Abstract

Intrinsic images are maps of surface properties, like depth, normal and albedo. 

I will show the results of simple experiments that suggest that very good modern depth, normal and albedo predictors are strongly sensitive to lighting – if you relight a scene in a reasonable way, the reported depth will change. This is intolerable. To fix this problem, we need to be able to produce many different lightings of the same scene. I will describe a method to do so. First, one learns a method to estimate albedo from images without any labelled training data (which turns out to perform well under traditional evaluations). Then, one forces an image generator to produce many different images that have the same albedo — with care, these are relightings of the same scene.  I will show some interim results suggesting that learned relightings might genuinely improve estimates of depth, normal and albedo.

But if an image generator can relight a scene, it likely has a representation of depth, normal, albedo and other useful scene properties somewhere.  I will show strong evidence that depth, normal and albedo can be extracted from two kinds of image generator, with minimal inconvenience or training data.   Furthermore, all these intrinsics are much less sensitive to lighting changes.  This suggests that the right way to obtain intrinsic images might be to recover them from image generators.  It also suggests image generators might “know” more about scene appearance than we realize.

About the speaker

I am currently Fulton-Watson-Copp chair in computer science at U. Illinois at Urbana-Champaign, where I moved from U.C Berkeley, where I was also full professor.  I have occupied the Fulton-Watson-Copp chair in Computer Science at the University of Illinois since 2014. I have published over 170 papers on computer vision, computer graphics and machine learning. I have served as program co-chair for IEEE Computer Vision and Pattern Recognition in 2000, 2011, 2018 and 2021, general co-chair for CVPR 2006 and 2015 and ICCV 2019, program co-chair for the European Conference on Computer Vision 2008, and am a regular member of the program committee of all major international conferences on computer vision.  I have served six years on the SIGGRAPH program committee, and am a regular reviewer for that conference. I have received best paper awards at the International Conference on Computer Vision and at the European Conference on Computer Vision. I received an IEEE technical achievement award for 2005 for my research.  I became an IEEE Fellow in 2009, and an ACM Fellow in 2014.  My textbook, “Computer Vision: A Modern Approach” (joint with J. Ponce and published by Prentice Hall) is now widely adopted as a course text (adoptions include MIT, U. Wisconsin-Madison, UIUC, Georgia Tech and U.C. Berkeley).  A further textbook, “Probability and Statistics for Computer Science”, is in print; yet another (“Applied Machine Learning”) has just appeared.   I have served two terms as Editor in Chief, IEEE TPAMI.  I have served on a number of scientific advisory boards.