{"id":2083,"date":"2022-12-06T13:10:35","date_gmt":"2022-12-06T18:10:35","guid":{"rendered":"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/?p=2083"},"modified":"2022-12-06T13:36:09","modified_gmt":"2022-12-06T18:36:09","slug":"marthascatmug-final-volumetric-video-of-hair","status":"publish","type":"post","link":"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/marthascatmug\/12\/06\/marthascatmug-final-volumetric-video-of-hair\/","title":{"rendered":"MarthasCatMug &#8211; Final: Volumetric Video of Hair"},"content":{"rendered":"<p><iframe loading=\"lazy\" title=\"hair\" width=\"840\" height=\"473\" src=\"https:\/\/www.youtube.com\/embed\/XI70N8oLHYo?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe><\/p>\n<p>For my final, I revisited my goal of capturing hair. I aimed to use photogrammetry in video form (also called &#8220;volumetric&#8221; or &#8220;4D&#8221; video) to try and capture moving hair. There were a lot of unknown factors going into the project that attracted me. I didn&#8217;t know how I was going to obtain many cameras, how I could set up a rig, how I could do the capturing process, or how I could process the many images taken by the many cameras into 3D models that could become frames. I wasn&#8217;t even sure if I&#8217;d get a good, bad, or an unintelligible result. I wanted the chance to do a project that was actually experimental and about hair though.<\/p>\n<p>In preparation for proposing this project I was looking into the idea\/concept of hair movement and on that subject, what I found were mostly technical art papers on hair simulation (ex. this <a href=\"https:\/\/google.github.io\/mediapipe\/solutions\/hair_segmentation\">paper<\/a> talks about obtaining hair motion data through clip in hair extensions). Artistically though, I found the pursuit of perfectly matching &#8220;real&#8221; hair through simulations a bit boring. I want the whimsy of photography and the &#8220;accuracy&#8221; of 3d models at the same time.<\/p>\n<figure id=\"attachment_2084\" aria-describedby=\"caption-attachment-2084\" style=\"width: 640px\" class=\"wp-caption aligncenter\"><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-2084 size-medium\" src=\"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/image-1-640x347.png\" alt=\"\" width=\"640\" height=\"347\" srcset=\"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/image-1-640x347.png 640w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/image-1-1024x555.png 1024w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/image-1-768x416.png 768w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/image-1-1536x832.png 1536w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/image-1-1200x650.png 1200w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/image-1.png 1921w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><figcaption id=\"caption-attachment-2084\" class=\"wp-caption-text\">21 photos, 18 aligned, very big model, about 425k vertices<\/figcaption><\/figure>\n<p>My process started with an exploration of the photogrammetry software: Agisoft Metashape which comes with a very useful 30-day free trial in the standard version. I experimented around with taking pictures and videos to get the hang of the software. My goal here was to see if I could find the fewest amount of photos (and therefore cameras) that would be needed to create a cohesive model. It turns out that number is somewhere just below 20 for a little less than 360 degree coverage.<\/p>\n<iframe width=\"640\" height=\"480\" src=\"https:\/\/sketchfab.com\/models\/335337a169e44e12bdc2e54a131b2b7f\/embed?autostart=0&autospin=0\" frameborder=\"0\" webkitallowfullscreen=\"true\" mozallowfullscreen=\"true\"><\/iframe>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-medium wp-image-2099\" src=\"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/borrowed-things-shzhang-640x426.png\" alt=\"\" width=\"640\" height=\"426\" srcset=\"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/borrowed-things-shzhang-640x426.png 640w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/borrowed-things-shzhang-1024x682.png 1024w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/borrowed-things-shzhang-768x512.png 768w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/borrowed-things-shzhang-1536x1023.png 1536w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/borrowed-things-shzhang-2048x1364.png 2048w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/borrowed-things-shzhang-1200x799.png 1200w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<p>I was able to borrow 18 Google Pixel phones (which all had 1\/8th, 240 ftps slow motion), 18 camera mounts, a very large LED light, several phone holders, a few clamps, and a bit of hardware from the Studio. I was then able to construct a hack-y photogrammetry setup.<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-medium wp-image-2101\" src=\"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/photogr-setup-shzhang-640x428.png\" alt=\"\" width=\"640\" height=\"428\" srcset=\"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/photogr-setup-shzhang-640x428.png 640w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/photogr-setup-shzhang-1024x684.png 1024w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/photogr-setup-shzhang-768x513.png 768w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/photogr-setup-shzhang-1536x1026.png 1536w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/photogr-setup-shzhang-2048x1368.png 2048w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/photogr-setup-shzhang-1200x802.png 1200w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<iframe width=\"640\" height=\"480\" src=\"https:\/\/sketchfab.com\/models\/52d8777bef344b2c8a9503445fc52f6f\/embed?autostart=0&autospin=0\" frameborder=\"0\" webkitallowfullscreen=\"true\" mozallowfullscreen=\"true\"><\/iframe>\n<p>Since the photogrammetry rig seemed pretty sound, the next step was to try using video. After filming a sample of hand movements, manually aligning the footage and exporting each video as folders of jpegs, I followed the &#8220;4D processing&#8221; <a href=\"https:\/\/agisoft.freshdesk.com\/support\/solutions\/articles\/31000155179-4d-processing\">Agisoft write-up<\/a>. This- no joke- took over 15 hours (and I didn&#8217;t even get to making the textures).<\/p>\n<p><iframe loading=\"lazy\" title=\"1\" width=\"840\" height=\"473\" src=\"https:\/\/www.youtube.com\/embed\/wsb0RDXVGJY?start=171&#038;feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe><\/p>\n<figure id=\"attachment_2086\" aria-describedby=\"caption-attachment-2086\" style=\"width: 640px\" class=\"wp-caption aligncenter\"><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-2086 size-medium\" src=\"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screenshot-2022-12-06-035941-640x454.png\" alt=\"\" width=\"640\" height=\"454\" srcset=\"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screenshot-2022-12-06-035941-640x454.png 640w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screenshot-2022-12-06-035941-1024x726.png 1024w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screenshot-2022-12-06-035941-768x544.png 768w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screenshot-2022-12-06-035941-1536x1089.png 1536w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screenshot-2022-12-06-035941-2048x1452.png 2048w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screenshot-2022-12-06-035941-1200x851.png 1200w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><figcaption id=\"caption-attachment-2086\" class=\"wp-caption-text\">manually synchronizing video<\/figcaption><\/figure>\n<figure id=\"attachment_2085\" aria-describedby=\"caption-attachment-2085\" style=\"width: 618px\" class=\"wp-caption aligncenter\"><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-2085 size-medium\" src=\"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screenshot-2022-12-06-035926-618x480.png\" alt=\"\" width=\"618\" height=\"480\" srcset=\"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screenshot-2022-12-06-035926-618x480.png 618w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screenshot-2022-12-06-035926-1024x796.png 1024w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screenshot-2022-12-06-035926-768x597.png 768w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screenshot-2022-12-06-035926-1536x1193.png 1536w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screenshot-2022-12-06-035926-1200x932.png 1200w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screenshot-2022-12-06-035926.png 1851w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><figcaption id=\"caption-attachment-2085\" class=\"wp-caption-text\">hand test, 720 frames (this was overkill)<\/figcaption><\/figure>\n<p>Aligning the photos took a few minutes (I was very lucky with this); generating a sparse point cloud took a bit over an hour; generating the dense point cloud took four; and generating the mesh took over 10. I didn&#8217;t dare try to generate the texture at that point because I was running out of time. I discovered here that I&#8217;d made a few mistakes:<\/p>\n<ol>\n<li style=\"text-align: left\">I forgot the setup I made is geared towards an upright object that is centered and not hands so this test was not the best to start with<\/li>\n<li style=\"text-align: left\">Auto focus :c<\/li>\n<li style=\"text-align: left\">Auto exposure adjustment :c<\/li>\n<li style=\"text-align: left\">Overlap should really be at about 70%+<\/li>\n<li style=\"text-align: left\">and &#8220;exclude stationary tie points&#8221; is an option that should only be checked if using turntable<\/li>\n<\/ol>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-2105\" src=\"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screen-Shot-2022-12-06-at-8.35.21-AM-640x380.png\" alt=\"\" width=\"332\" height=\"197\" srcset=\"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screen-Shot-2022-12-06-at-8.35.21-AM-640x380.png 640w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screen-Shot-2022-12-06-at-8.35.21-AM-1024x608.png 1024w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screen-Shot-2022-12-06-at-8.35.21-AM-768x456.png 768w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/Screen-Shot-2022-12-06-at-8.35.21-AM.png 1044w\" sizes=\"(max-width: 332px) 85vw, 332px\" \/><\/p>\n<p><iframe loading=\"lazy\" title=\"textureless hand test vol vid\" width=\"840\" height=\"473\" src=\"https:\/\/www.youtube.com\/embed\/DYIwqgFjDcs?start=44&#038;feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe><\/p>\n<p>So, what next? Cry ? yes :C but also I did try to wrangle the hair footage I have into at least a sliver of volumetric capture within the time I had.<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\" wp-image-2104 aligncenter\" src=\"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/IMG_1894-640x480.jpg\" alt=\"\" width=\"463\" height=\"347\" srcset=\"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/IMG_1894-640x480.jpg 640w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/IMG_1894-1024x768.jpg 1024w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/IMG_1894-768x576.jpg 768w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/IMG_1894-1536x1152.jpg 1536w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/IMG_1894-2048x1536.jpg 2048w, https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-content\/uploads\/2022\/12\/IMG_1894-1200x900.jpg 1200w\" sizes=\"(max-width: 463px) 85vw, 463px\" \/><\/p>\n<p>I think that in a more complete, working, long form, I&#8217;d like for my project to live in Virtual Reality. Viewing 3D models on a screen is nice but I think there is a fun quality of experience in navigating around virtual 3D objects. Also, I guess my project is all about digitization: taking information from the physical world and not really returning.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>For my final, I revisited my goal of capturing hair. I aimed to use photogrammetry in video form (also called &#8220;volumetric&#8221; or &#8220;4D&#8221; video) to try and capture moving hair. There were a lot of unknown factors going into the project that attracted me. I didn&#8217;t know how I was going to obtain many cameras, &hellip; <a href=\"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/marthascatmug\/12\/06\/marthascatmug-final-volumetric-video-of-hair\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;MarthasCatMug &#8211; Final: Volumetric Video of Hair&#8221;<\/span><\/a><\/p>\n","protected":false},"author":17,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[20],"tags":[],"_links":{"self":[{"href":"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-json\/wp\/v2\/posts\/2083"}],"collection":[{"href":"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-json\/wp\/v2\/users\/17"}],"replies":[{"embeddable":true,"href":"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-json\/wp\/v2\/comments?post=2083"}],"version-history":[{"count":7,"href":"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-json\/wp\/v2\/posts\/2083\/revisions"}],"predecessor-version":[{"id":2272,"href":"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-json\/wp\/v2\/posts\/2083\/revisions\/2272"}],"wp:attachment":[{"href":"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-json\/wp\/v2\/media?parent=2083"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-json\/wp\/v2\/categories?post=2083"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/courses.ideate.cmu.edu\/60-461\/f2022\/wp-json\/wp\/v2\/tags?post=2083"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}