Portrait mode sucks

Barks · 28 Sep 2023 at 14:38

So this post is a bit of a rant but I am hoping someone who knows more than me about optical engineering can offer some insight.

The Portrait mode feature on iPhones and other smart phones to me looks completely unnatural. As this has just been a ‘gimmick’ on smartphones over the last few years it was fine but now I noticed during the Rugby World cup they were clearly using footage with the background artificially blurred. It looked awful. You could literally see parts of the player‘s top becoming blurred.

When you take a photo with a camera and focus on the subject, the background will be blurred at different degrees depending on the depth. A pin sharp 1.8f portrait will have the face razor sharp in focus, the horizon completely blurred and other closer elements slightly blurred. When you use AI to just blur the background it loses that sense of depth. It’s a binary IN or OUT of focus.

I honestly think it looks horrific and I accept it’s a fun gimmick on smartphones but I’m annoyed to see it is now manifesting itself in commercial applications.

The thing is even the standard lens in my 12 Pro Max takes great photos with bokeh WITHOUT the artificial manipulation.

Take delivery of the 15 Pro Max tomorrow so maybe they’ve improved it but AI cannot possibly manually adjust different elements of focus based on depth from a 2D image so it’s never going to look good… this is where I was wondering if anyone could offer insight?

Raymond Lin · 28 Sep 2023 at 14:55

It is getting better but the biggest problems with it are

1 - Isolation / Selection - correctly selecting what is in focus and what is not. The edges and do it accurately
2 - Fall off / Transition - AI does it too abruptly, real lenses and real optics don't do that. Out of focus areas gradually gets more blur as it is further away.
3 - The illusion of the bokeh - different optics produces different kind of bokeh. Rather than Gaussian blur the whole thing.

I just don't use it, tried it before and it's just not needed. The average person who uses it think it looks great because it is all seen in movies, shot with a $30,000 cine lenses and they just want the effect.

iPhone 14 Pro.

SDK^ · 28 Sep 2023 at 18:09

The depth maps and software algorithms are improving.
For social media, viewed on a phone/iPad display, at arms length the current Portrait mode outputs are fine.

Barks · 28 Sep 2023 at 22:48

SDK^ said:
The depth maps and software algorithms are improving.
For social media, viewed on a phone/iPad display, at arms length the current Portrait mode outputs are fine.

I’m not sure they are though, they still stand out like a sore thumb to me. Just because the images on Insta, for example, are heavily compressed it doesn’t mean you can’t appreciate which ones were taken on a proper camera.

Barks · 28 Sep 2023 at 22:50

Raymond Lin said:
It is getting better but the biggest problems with it are

1 - Isolation / Selection - correctly selecting what is in focus and what is not. The edges and do it accurately
2 - Fall off / Transition - AI does it too abruptly, real lenses and real optics don't do that. Out of focus areas gradually gets more blur as it is further away.
3 - The illusion of the bokeh - different optics produces different kind of bokeh. Rather than Gaussian blur the whole thing.

I just don't use it, tried it before and it's just not needed. The average person who uses it think it looks great because it is all seen in movies, shot with a $30,000 cine lenses and they just want the effect.

iPhone 14 Pro.

Now do the same photo on the Olympus of your iphone and the what looks like a prolytic digestive enzyme shake!

Raymond Lin · 28 Sep 2023 at 23:17

Barks said:
Now do the same photo on the Olympus of your iphone and the what looks like a prolytic digestive enzyme shake!

I can do you one better.

Same photo, taken with Sony A7R3 (42mp), with Sigma 24-70/ DG DN at 2.8. at 48.4mm (I was going for 50mm).

SDK^ · 29 Sep 2023 at 09:14

Here is a quiz for you.
Two sets of photos I just took this morning - using an iPhone 13 Pro Max in Portrait Mode and a Sony A73 with an F2.8 lens.

Q : Images 1 & 2 : Which were taken using the iPhone - A or B ?

1:

2:

Barks · 30 Sep 2023 at 11:39

I reckon B is iPhone but you’re not making it easy with those compositions. Intrigued to hear the result!

Quartz · 30 Sep 2023 at 11:53

A friend of mine used to run an advertising company. Long retired now, of course. On one shoot some of his staff were arguing about camera equipment needed to take good photos. The photographer they'd hired for the shoot was world-famous and overheard this. He said that he would take a set of photos on an instamatic (or whatever the basic 35mm camera of the day was in the 70s or 80s) - no fancy lenses, no nothing - and if at least three were not usable then he would work for free. The photos were duly developed, the pictures examined, and his fee was paid in full.

SDK^ · 30 Sep 2023 at 16:14

Barks said:
I reckon B is iPhone but you’re not making it easy with those compositions. Intrigued to hear the result!

Shot A in both is the iPhone

The idea for the composition was to show the depth layers to test how the iPhone portrait mode software compares to the optical bokeh of the Sony camera & lens

Greboth · 2 Oct 2023 at 10:39

That's interesting to see and at that size it's very hard to tell the difference. Side by side, it's easy to see in the second picture A is the phone due to the darker green bush behind and branch sticking up being blurred but not in B as it's not too far removed from the plane of focus. Is that enough to noticed though without having a side by side? and I can't see such an issue with the first.

Calin Banc · 7 Oct 2023 at 22:19

Quartz said:
A friend of mine used to run an advertising company. Long retired now, of course. On one shoot some of his staff were arguing about camera equipment needed to take good photos. The photographer they'd hired for the shoot was world-famous and overheard this. He said that he would take a set of photos on an instamatic (or whatever the basic 35mm camera of the day was in the 70s or 80s) - no fancy lenses, no nothing - and if at least three were not usable then he would work for free. The photos were duly developed, the pictures examined, and his fee was paid in full.

Image you hire someone to shoot your wedding. Instead of using the best gear he can, comes with a cheap phone or cheap gear. The photos are probably decent (if he's VERY good), but not as good as they could be otherwise and some images were not even taken, because it was impossible with the equipment he had. Would you be happy with that? I won't...

Gear doesn't matter is a very simplistic and wrong way to see things. Cinema cameras wouldn't exist, phones would be used everywhere. Clearly that's not the case, even when the guy behind is exceptionally good at his job.

Barks · 8 Oct 2023 at 06:22

Calin Banc said:
Image you hire someone to shoot your wedding. Instead of using the best gear he can, comes with a cheap phone or cheap gear. The photos are probably decent (if he's VERY good), but not as good as they could be otherwise and some images were not even taken, because it was impossible with the equipment he had. Would you be happy with that? I won't...

Gear doesn't matter is a very simplistic and wrong way to see things. Cinema cameras wouldn't exist, phones would be used everywhere. Clearly that's not the case, even when the guy behind is exceptionally good at his job.

If I hired a videographer who shot my wedding on cinematic mode or snapped photos with artificial backgrounds I wouldn’t just be unhappy I would probably throttle them

They’re using cinematic mode footage during the Rugby World Cup to capture people in the crowd and it looks truly and utterly awful.

Chuk_Chuk · 8 Oct 2023 at 07:32

Greboth said:
That's interesting to see and at that size it's very hard to tell the difference. Side by side, it's easy to see in the second picture A is the phone due to the darker green bush behind and branch sticking up being blurred but not in B as it's not too far removed from the plane of focus. Is that enough to noticed though without having a side by side? and I can't see such an issue with the first.

With the first set of images I was comparing the amount of blur on the standalone cars to the right with the row of cars on the left.

With image A certain parts of the car looked more out of focus compared to the other car in its depth plane. However I am on a mobile viewing these so that may have an affect.

Minstadave · 8 Oct 2023 at 08:57

Raymond Lin said:
iPhone 14 Pro.

Everything is a trade off. Phones are very close to "good enough" that the benfit of not having to lug around a camera outweighs the improve picture quality for the majority of people/uses.

Barks · 9 Oct 2023 at 07:15

Minstadave said:
Everything is a trade off. Phones are very close to "good enough" that the benfit of not having to lug around a camera outweighs the improve picture quality for the majority of people/uses.

"Good enough" still looks awful and completely artificial. Most of the times I've seen people use portrait mode on the iPhone, the natural bokeh from the primary lens would have looked much better anyway, but the portrait mode version just butchers what would have been a nice photo. Even my 12 Pro Max got decent portrait shots with subtle and natural bokeh (not on portrait mode).

I can't stand it. It's one of those things we'll look back on and laugh at ourselves. I doubt Steve Jobs would even have signed it off, it's just not good enough for an Apple product, not in the first iteration, not in this one either. If I could remove it entirely from my 15 Pro Max so it doesn't even appear as an option then I would. Naff, tacky, awful (in my humble opinion, I respect yours to disagree!)

LiE · 9 Oct 2023 at 14:51

I find reducing the blur to look more subtle gives the best results. I hate the cut out look some of the default portrait modes use.

Barks · 10 Oct 2023 at 19:43

LiE said:
I find reducing the blur to look more subtle gives the best results. I hate the cut out look some of the default portrait modes use.

Yes I agree with this completely. They do look better when you crank the background blur down. The default option looks really artificial.

mrk · 13 Oct 2023 at 23:05

It's fine on Samsung, the depth sensor does a decent job I find. I always use portrait mode at the lower blur values, 1-3 max as that feels the most natural.

Guess which is portrait mode and which is normal mode:

SDK^ · 14 Oct 2023 at 18:21

mrk said:
It's fine on Samsung, the depth sensor does a decent job I find. I always use portrait mode at the lower blur values, 1-3 max as that feels the most natural.

Guess which is portrait mode and which is normal mode:

Objects like that are generally easy for portrait mode.
It usually struggles around branches, hair or when the background appears between legs & arms and it doesn't blur those sections.
e.g. As in my example above, where portrait mode is blurring branches (photo 2A), when they are actually on the plane of focus (photo 2B Mirrorless shot)