Gaussian blur inconsistencies #279
Comments
Relates to #279. Thank you to @softhorizons for discussing this!
|
Hello @softhorizons. Thank you for your detective work! Your sample image and reproducible steps were a huge help in figuring out how to address this. A gaussian blur radius value has to be passed through some (convoluted) math equations to ultimately produce the lambda value that PhotoDemon requires for its "precise" gaussian mode. This value modifies the shape of the gaussian kernel, which in turn determines how much weight each pixel is assigned during various blur stages. I have modified a step in PhotoDemon's equations to use a formula from the public-domain ImageJ scientific imaging library (https://imagej.nih.gov/ij/source/ij/plugin/filter/GaussianBlur.java). The ImageJ authors claim this is "the same as in Photoshop" and indeed, the newly modified PhotoDemon "precision" algorithm produces quite a bit stronger blur - closer to PD's "fast" estimation mode, as well as Photoshop's Gaussian Blur tool. The exact reasoning behind their formula is currently over my head, but the similarity of results between PD's "fast" mode (which I know is good, at least for the approximation it provides), Photoshop, and a variety of other photo editors makes me comfortable introducing this change despite not being 100% sure of the math behind it. Whenever convenient, would you mind taking a look at the latest nightly build and letting me know if it meets your expectations? Thank you! |
|
Unfortunately I don't have the chops to comment on the math either. I observe that, with your update, Precise mode is now a bit more aggressive than Fast, e.g. result of Fast @ 21 looks similar to Precise @ 18 or 19 on the test image. As for Photoshop, its visually equivalent setting is around 6! That leaves me a little uneasy, so I made this test image. This said, the definition of Gaussian blur radius seems to be a matter of some controversy. It all depends where you draw the line. I've seen naive implementations via straight convolution disagree on their choices of convolutions kernels: some authors in effect use a radius 4 larger than the classic one since the first and last 2 terms are tiny once you get past a radius of 3 or 4. |
|
Thank you for the quick reply!
Yes, this is exactly as expected. The two methods converge (asymptotically) as radius approaches infinity. At extremely low radii, the "fast" method is very imprecise. (The "fast" method just uses iterative integer-based box blurs to approximate a gaussian, and at a radius of 5, it has very little to work with.) This is generally not an issue as a precise gaussian filter is extremely fast at small radii - and indeed, the "fast" mode really only exists to ease performance woes when the radius value gets extremely large. (Or when a user just needs some kind of blur and they don't care about the details, like producing a drop shadow.)
It is, and Adobe often makes things worse because they like to adopt their own definitions for everything (definitions that can change from version to version, no less). The only copy of PS I own is version 5 (20+ years old now!) and its gaussian blur tends to be twice as strong as other software, so a radius of 10.5 looks nearly identical to PhotoDemon's at 21 - a good sign, I figure, even though newer versions of PS may be different. The trouble with gaussian blurs is that "radius" isn't applicable to the blur at all, because a gaussian kernel is infinite - it's just that the edges get so small at some point that you can ignore them without meaningfully affecting the output, like you say. This stackoverflow page has a nice breakdown of the specifics; to quote the answer given there:
This could explain Photoshop's behavior. It likely has different needs since they support things like 128-bit HDR images that possess much more fine detail than a standard 32-bit SDR image - so their tool's relationship between radius and sigma could reasonably differ from a simpler editor like PhotoDemon, and everyone involved could be "right", so to speak. From what I can tell, ImageJ's sigma calculation (which I dropped into PD's "precise" algorithm) is a good fit for PD because it treats the radius as the point where the kernel values are too small to affect standard 8-bit color values. This is a reasonable compromise for a 32-bit image editor like PhotoDemon, and the results are nearly identical to other 32-bit editors like Paint.NET. (GIMP looks closer to my old copy of Photoshop - cut the radius in half, to 10.5, and the results are very close, although I believe GIMP performs the blur in a gamma-corrected space - yet one more wrinkle!) 🤷♂ Such is life as an open-source developer, ha. There are never good answers - only confusing ones! I feel good about the current relationship between PD's fast and precise modes (as I can't really force those to converge better than they do now, and I believe both modes have value), so my remaining question is: what to do about the radius differences between us and Photoshop? I could easily double the strength of PD's gaussian blur to more closely approximate GIMP/Photoshop's output, but I feel like I need to do more research to figure out if this is worthwhile. Generally speaking, I like to err on the side of mimicking Photoshop (even when I disagree with it) because it makes it easier for users to use Photoshop tutorials in PhotoDemon without egregious modifications. But it's a fuzzy line, especially when I don't fully understand why they've chosen to do things the way they do... I guess I'm saying I'm open to suggestions either way. (Apologies that I don't have a firmer opinion on this one!) |
Relates to #279 I haven't forgotten about this! It's just taken me some time to study the math involved and figure out a proper plan of attack. In this commit, I've reworked the way PD's fastest gaussian approximation filter works. Thank you to Ivan Kutskir and this link for advice on how to handle this: http://blog.ivank.net/fastest-gaussian-blur.html After trying a bunch of different approaches, his is the one I like the most - because the math is approachable (yay!), and also because the similarity to Photoshop is really, *really* good. As a byproduct of this, PD's "fast" and "precise" modes now produce much more similar results under most radii. I am now going to attempt to rework edge handling for this same algorithm, so that both it and the IIR filter use the same technique.
Relates to #279 This new gaussian blur design takes advantage of the perf improvements in f010eec and extends them to the vertical component as well. (Actually, we cheat and perform the vertical blurs by rotating the image 90 degrees, applying horizontal blurs, then rotating it back - this provides a larger perf boost, and it ensures that vertical edge behavior is 100% identical to horizontal edge behavior.) Edge pixel handling is now identical to PD's more precise IIR-based technique, with edge pixels being "extended" instead of "clamped". This produces closer results to Photoshop, for better or worse. I've also further reduced memory requirements of the fast function (alongside some truly ugly object aliasing, lol), and performance of PD's fast gaussian is now back on par with Paint.NET.
Relates to #279 PD's IIR gaussian blur implementation is derived from a 1994 paper by Luis Alvarez and Luis Mazorra: https://www.jstor.org/stable/2158018?seq=1#page_scan_tab_contents The actual code is based on a 2012 implementation of said paper by Pascal Getreuer (BSD-licensed): https://web.archive.org/web/20121128102249/http://www.getreuer.info/home/gaussianiir In 2013, Pascal published an updated paper that compared various gaussian estimation techniques, particularly their performance and accuracy vs a "true" gaussian: http://www.ipol.im/pub/art/2013/87/?utm_source=doi In that paper, he proposed a number of minor modifications to the original Alvarez-Mazorra approach, including: - "In the original work [12], Alvarez and Mazorra set q = σ. While the method with q = σ converges to Gaussian filtering with parameter σ as K → ∞, this choice tends to undersmooth for small K. Here we introduce an adjustment to compensate for this effect..." - "Alvarez and Mazorra’s discussion is on the infinite grid and does not include boundary handling. Here we develop its use with half-sample symmetric boundaries." Pascal also provided C source code with his changes (still BSD-licensed). I have now rewritten his modified version in VB6, along with a number of VB6-specific optimizations. This improves accuracy vs the original implementation (particularly along boundaries), reduces running time (by ~10-20%), and greatly reduces memory usage (75% less memory required because we process channels one-at-a-time using a shared buffer). These are all valuable improvements, and I'm excited to have them. That said, the Alvarez-Mazorra method is not necessarily the best choice for accuracy vs a "true" gaussian. Even with multiple iterations, its error remains relatively high - somewhere between 8 and 10% for all radii > 5, with little gain from increasing the iterations beyond 3 (or even 2). This means that @softhorizons detective work in #279 was absolutely correct, and an iterative box blur is actually *more* precise than this particular IIR implementation, even with larger radii. I will shortly be updating PhotoDemon's UI in various blur tools to account for this discrepancy. Pascal's paper provides two alternate, more accurate gaussian estimations (ignoring a DCT-based one, which would be near-perfect in accuracy but a performance nightmare)... but wow, the math for these is *ugly*. I may take a stab at one out of morbid curiosity, but it'll likely take me some time, as the more accurate techniques require a lot of complex number manipulation (which VB6 obviously doesn't provide natively, so lucky me, I'd get to implement it from scratch 😬 )
Relates to #279 Well, this is one of the ugliest C translations I've done in awhile. Many thanks to Pascal Getreuer for his excellent paper, "A Survey of Gaussian Convolution Algorithms": http://www.ipol.im/pub/art/2013/87/ In that paper, Pascal confirms what was suspected in #279 - that the Alvarez-Mazorra algorithm used by PD's IIR gaussian implementation can deviate strongly from a true gaussian (10+%, even at 3rd-order!). This is unacceptable, alas. Fortunately, the same paper suggests an excellent replacement: another time-invariant IIR approach based on Deriche's 1993 paper, "Recursively implementing the Gaussian and its derivatives": https://hal.inria.fr/file/index/docid/74778/filename/RR-1893.pdf Pascal adds a number of important updates to Deriche's original method, including significant improvements to edge-handling. Critically, he also went to the trouble of calculating deviations from a true gaussian for all surveyed approximators, and Deriche's was confirmed to be the most accurate of the bunch, with a deviation of well under 1% at third-order, and less than 0.1% for 4th-order. (PD's implementation provides 2nd, 3rd, and 4th order implementations, with a current default to 3rd order.) Like the old IIR method, Deriche's performance is radius-invariant, with no meaningful difference between a radius of 1 or a radius of 500. I have now modified PD's Effects > Blur > Gaussian Blur tool to use the Deriche algorithm when the "precise" setting is selected. I may update this tool's UI to directly expose all estimation algorithms, as they all have trade-offs, and it may be worthwhile to allow users to increase the order for AM and Deriche's approximators in particular (for when extreme quality is required). The new Deriche algorithm is slightly slower than PD's AM algorithm, and about 3x slower than PD's "fast" iterative box blur algorithm. However, it's still several orders of magnitude faster than a "true" gaussian, with imperceptible differences in standard sRGB space. Many thanks to @softhorizons for their original detective work on this topic, and for their patience while I worked on a solution.
|
Hello @softhorizons . I apologize that it's taken me awhile to follow up on this. Amidst other work, I've finally studied a number of gaussian resources so I can bring PhotoDemon's blur tool into "alignment" with Photoshop's (or at least understand enough math to make an informed decision about what needs fixing!). I've recently committed two all-new gaussian blur algorithms, and I wanted to run them by you before closing this issue. PhotoDemon's As an added bonus, the "fast" and "precise" methods now produce very similar results to each other across a wide range of radii, despite using totally different algorithms. The "fast" method is now 2x - 3x faster than before, while also using much less memory, while also using a redesigned algorithm that handles edge pixels identically to Photoshop. In fact, if you simply divide Photoshop's radius value by 2 when comparing the results, PhotoDemon now produces near-identical results across most images, which makes me think I've finally figured out how Photoshop's tool works. PD's "precise" method now uses a totally different algorithm derived from the excellent paper, A Survey of Gaussian Convolution Algorithms" by Pascal Getreuer. In that paper, Pascal confirms exactly what you suspected - that PhotoDemon's old "precise" technique was not as precise as I claimed to be. (Specifically, I used the Alvarez-Mazorra method described in the paper, which can apparently deviate from a true gaussian by more than 10%!) I didn't realize this before, and I apologize that I didn't study it more closely. For maximum accuracy, the paper recommends an alternate approach based on this 1993 paper by Rachid Deriche. That approach is waaaay more complicated (requiring a bunch of insane math involving imaginary numbers), but it's also radius-invariant (meaning processing time does not increase with radius), and the results are incredibly accurate - it deviates from a true gaussian by less than 1% as a 3rd-order filter, and less than 0.1% as a 4th-order. I have just finished implementing both the 3rd- and 4th-order techniques in PhotoDemon, although only the 3rd-order is currently used by the Gaussian Blur tool. (I may add a toggle for this in the coming days.) Using your original photo at the top of this issue, here is the final result from the new tool. Top is the "fast" method, bottom is the "precise" method. Radius is 21 pixels:
For comparison, here is Photoshop's result at 10.5 pixels: Pretty good, I think! I am very excited that PD's "fast" option is now much faster, and near-identical to Photoshop, while the "precise" setting is actually what it claims to be (very very precise!). I would never have improved this tool without your help. Thank you. Please let me know if I can include you in PhotoDemon's list of contributors, and what name and/or URL you'd like me to use. Also, if you notice any problems with the new methods, please let me know and I'll investigate. Thank you! |
|
Thank you for looking into this, Tanner. The new samples look great!
To my eye, the “precise” implementation looks slightly closer to PhotoShop, yet PS is crazy fast. I’m guessing that they’re using DCTs implemented on the GPU and/or x86 MMX/SIMD hardware. This suggests you might want to look at something like libjpeg-turbo if you ever find yourself with spare time on your hands.
As several authors have conjectured over the years, the large discrepancy vs. PhotoShop looks like it’s likely just a UI decision by Adobe, mathematical precision be damned. The fact that the factor is such a nice, even 2.0x seems to support that conclusion. This has been obscured in the past when PhotoShop was compared to the generally inaccurate implementations that are floating around.
Note that the 2x factor has ramifications for your own UI. I believe that you generally try to maximize PD’s compatibility with PhotoShop to maximize the usefulness of PS “recipes,” making the large 2x discrepancy hard to just ignore. Off the top of my head, you might consider offering one of: explanatory text in the dialog box, a tooltip with explanatory text, or 2 parallel radiuses in the radius input control (one regular and one PhotoShop mode) with the 2 controls slaved to each other so that changing one recomputes the other.
If you’d like to add me to the contributor list, it’s:
John Desrosiers https://johndesrosiers.com
Keep up the amazing work! JD
From: Tanner
Sent: Thursday, November 07, 2019 10:21 AM
To: tannerhelland/PhotoDemon <PhotoDemon@noreply.github.com>
Cc: softhorizons <johndesr@gmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [tannerhelland/PhotoDemon] Gaussian blur inconsistencies (#279)
Hello @softhorizons <https://github.com/softhorizons> . I apologize that it's taken me awhile to follow up on this.
Amidst other work, I've finally studied a number of gaussian resources so I can bring PhotoDemon's blur tool into "alignment" with Photoshop's (or at least understand enough math to make an informed decision about what needs fixing!). I've recently committed two all-new gaussian blur algorithms, and I wanted to run them by you before closing this issue.
PhotoDemon's Effects > Blur > Gaussian Blur tool is now running on new versions of both algorithms ("fast" and "precise"). A lot has changed, but the short story is that the "fast" setting now produces near-identical results to Photoshop, while the "precise" setting produces near-identical results to exact gaussian blur implementations.
As an added bonus, the "fast" and "precise" methods now produce very similar results to each other across a wide range of radii, despite using totally different algorithms.
The "fast" method is now 2x - 3x faster than before, while also using much less memory, while also using a redesigned algorithm that handles edge pixels identically to Photoshop. In fact, if you simply divide Photoshop's radius value by 2 when comparing the results, PhotoDemon now produces near-identical results across most images, which makes me think I've finally figured out how Photoshop's tool works.
PD's "precise" method now uses a totally different algorithm derived from the excellent paper, A Survey of Gaussian Convolution Algorithms <http://www.ipol.im/pub/art/2013/87/> " by Pascal Getreuer. In that paper, Pascal confirms exactly what you suspected - that PhotoDemon's old "precise" technique was not as precise as I claimed to be. (Specifically, I used the Alvarez-Mazorra method described in the paper, which can apparently deviate from a true gaussian by more than 10%!) I didn't realize this before, and I apologize that I didn't study it more closely.
For maximum accuracy, the paper recommends an alternate approach based on this 1993 paper by Rachid Deriche <https://hal.inria.fr/inria-00074778/document> . That approach is waaaay more complicated (requiring a bunch of insane math involving imaginary numbers), but it's also radius-invariant (meaning processing time does not increase with radius), and the results are incredibly accurate - it deviates from a true gaussian by less than 1% as a 3rd-order filter, and less than 0.1% as a 4th-order. I have just finished implementing both the 3rd- and 4th-order techniques in PhotoDemon, although only the 3rd-order is currently used by the Gaussian Blur tool. (I may add a toggle for this in the coming days.)
Using your original photo at the top of this issue, here is the final result from the new tool. Top is the "fast" method, bottom is the "precise" method. Radius is 21 pixels:
<https://user-images.githubusercontent.com/1930029/68414888-47172f00-014e-11ea-8bca-e265949d1130.png>
<https://user-images.githubusercontent.com/1930029/68414935-6dd56580-014e-11ea-83b7-5e71c15213ee.png>
For comparison, here is Photoshop's result at 10.5 pixels:
<https://user-images.githubusercontent.com/1930029/68415373-78442f00-014f-11ea-9ec3-ee19ea10ae2d.jpg>
Pretty good, I think!
I am very excited that PD's "fast" option is now much faster, and near-identical to Photoshop, while the "precise" setting is actually what it claims to be (very very precise!).
I would never have improved this tool without your help. Thank you. Please let me know if I can include you in PhotoDemon's list of contributors, and what name and/or URL you'd like me to use.
Also, if you notice any problems with the new methods, please let me know and I'll investigate. Thank you!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#279?email_source=notifications&email_token=ACAKYMT53RGTHI3FK24XSWTQSRMA3A5CNFSM4IY6WSRKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDNKYVI#issuecomment-551201877> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACAKYMXPK3EQCJULKHEHZNLQSRMA3ANCNFSM4IY6WSRA> . <https://github.com/notifications/beacon/ACAKYMUOP5JOFMKCV77OPK3QSRMA3A5CNFSM4IY6WSRKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDNKYVI.gif>
|
Relates to #279 As usual, trying to match the output of other software is a fool's errand... but that doesn't mean I can't try! PD's gaussian blur tool now offers a plethora of options (and of course, progressive disclosure is used to minimize complexity for users who don't need/care about this level of control). The gaussian blur "radius" selector can now be toggled between three different modes: standard radius, Photoshop-equivalent radius (basically a 2x multiplier), or bare gaussian sigma (used by GIMP and some scientific tools). PD internally converts between the three measurements so the user doesn't need to worry about details; whichever measurement you use, the end result will be mathematically correct. Similarly, PD now offers three quality settings (instead of just two): fast, precise, and "custom". When selected, "custom" mode exposes additional toggles for specific algorithms (iterative box blur, Alvarez-Mazorra anisotropic diffusion, and Deriche's IIR), as well as iteration count. (Most approximations produce better results with more iterations.) This should make it possible to mimic just about any other photo editor, whether photography- or science-oriented. Also included in this commit are perf improvements to PD's ultra-accurate Deriche gaussian implementation. It is now very close, performance-wise, to my Alvarez-Mazorra implementation - but with significantly improved accuracy vs a "true" gaussian. This is a nice win, especially at enormous radii. While I was here, I also doubled the maximum radius exposed by the UI. Blur up to 1000px if you want; all algorithms are radius-invariant (minus minor initialization costs), so radius does not affect performance
|
Oh boy, December and half of January got away from me! Apologies that I haven't followed up here in a timely fashion, despite the work being completed back in November. First up, @softhorizons has been added to the program's contributor list. Thank you again for your patience and help, John. 🙏 After prototyping a few different approaches, I settled on just providing a toggle for the three different blur measurements: radius, Photoshop radius, and standard deviation. This provides compatibility with all other software (I hope!) without messing up existing PhotoDemon macros that use a standard radius measurement. It also keeps things simple for users who don't know/don't care. Besides "fast" and "precise" measurements (both of which are faster and more precise than before), I've also added a new "custom" option where the user can select a specific blur algorithm and iteration count. This is almost certainly overkill, but maybe someone out there will find it useful! (And because those advanced options are hidden by default, it again won't bother those who don't know/don't care.)
I learned a great deal about gaussian blur theory from this project, and PhotoDemon's open-source license means other developers can reuse all of this work in their own projects. One step closer to a Photoshop-esque experience for the common man! 😄 I'm closing this issue, but if anyone has further knowledge of how to improve PhotoDemon's gaussian blur tool(s), please let me know. |
Let's start with version information
Please describe the bug
Large inconsistency between Fast and Precise blur
Expected behavior
Expected virtually identical results between Fast and Precise blur. Source code claims 3% max theoretical difference. However, the attached example shows that Fast visibly yields much more blur. I'm not sure whether Fast or Precise is more correct, though I suspect Fast is the more correct one (see below).
Note that this is an important use case since Gaussian blur is commonly used to descreen scans/photos of halftoned photos.
How to reproduce the bug
The text was updated successfully, but these errors were encountered: