Archive for March, 2010

The music of Paris 26 Gigapixels

Wednesday, March 17th, 2010

Many of you sent us messages about the music used on the website of Paris 26 Gigapixels. Some of you recognized it: La Valse d’Amélie is part of the soundtrack of the movie “Amelie” (Le Fabuleux Destin d’Amélie Poulain). The author is the French musician and composer Yann Tiersen.

> Find out more about this movie

Paris 26 Gigapixels is now online!

Thursday, March 11th, 2010

Update: the DNS are still propagating and you may need to wait a few hours before you can access the website, depending on your location and internet provider.

Making of: step 3, the rendering

Thursday, March 11th, 2010

Last step before putting the image on-line: the rendering. The rendering is the process where the final image is actually generated in Autopano Giga after the panorama edition.

Thanks to Intel, we were able to take advantage of powerful hardware to render the image in record time. Let’s ponder a bit: we borrowed an Intel Server System SR2600UR that included 2 Intel Xeon processors 5500 series and 6 SSD hard drives of 160 GB each, allowing a much faster data writing than standard hard drives. With this hardware that would have certainly made a number of geeks jealous (16 cores, 24 GB RAM, 1 TB SSD), the rendering lasted only 3 hours and 14 minutes: an amazing performance. Two years ago for the record of Harlem 13 Gigapixels, we needed 48 hours for the rendering to complete; the image was two times smaller. You can read more about this in the case study written by Intel.

But the hardest part is yet to come…

It is true that, thanks to a good hardware and software that can exploit all available power, it is rather easy to create large images such as Paris 26 Gigapixels. But the necessary adjustments that have to be made thereafter can be numerous:
- First, the quality of the stitching was very good and there was virtually no assembly errors in the image of Paris.  But we had to better harmonize the colors in some areas of the sky. This is a real technological challenge.
- Second, our lawyer had the excellent idea of advising us to mask any areas that could be an invasion of privacy, including license plates and recognizable faces! We had to split the view into smaller pieces that could be edited in Photoshop. Note: Photoshop is internally limited to a size of 300,000 by 300,000 pixels. So the image of Paris could not be opened directly. I take this opportunity here to make an appeal to John Nack: “Hey John, in the next Photoshop, please raise this internal limit to 3,000,000, for example. ;) Thanks John!

We embarked on this great job of blurring or masking the said areas, something that is not so easy when the image you are working on is the size of two football fields! We developed a tool that can cut, and then easily reassemble such large images. It will soon be made available to the Autopano community.

We took advantage of the required retouching step to include a few surprises in the picture so that you can also have fun searching for little details everywhere! There are 10: we  let you discover them by yourself :) What? Easter eggs? Yes sir!

Along with the editing work, the website has been prepared and asks nothing more than to receive this giant image in its two versions:
- The default version, in Flash format, which includes interactive elements like an automatic visit. The Flash viewer used is KRPano, the same technology we use in our product Autopano Tour.
- The version in HDview format, accessible via the top-right button, does not offer the interactive features but offers a smoother experience while navigating inside the image.

Finally, we had to cope with one last issue: offer this experience to everyone regardless of server load. This website is made up from 600,000 files indeed. Our partner IPSyn helped us in geolocalized deployment on many servers around the world. With this, we should take the load :)

Have a nice visit to Paris and see you soon for new gigapixel adventures.

Making of: step 2, the stitching of Paris

Thursday, March 11th, 2010

The first step after the shooting is to cope with RAW files. Let Arnaud Frich explain his method and goal :

The stitching itself remains a bit complicated with a project of this size. Even if using a motorized panoramic head helps a lot during the shooting, the stitching still doesn’t give perfect results because of certain points : first, robustness of the motorized panoramic head (we had a break after each row to manually correct the focus) and second, repeating patterns in some of the buildings.

Let me remind you of the shooting layout in the Paris image : 138 columns by 17 rows, that is 2346 images. The first stitching gave the following image :

We can notice a shift in the rows which comes from a shooting issue: we missed 3 images during the shooting at the edge of one row and the beginning of the next. It’s the kind of stuff you don’t notice when out in the field but when you come back to the office (BTW: it was really a nightmare figuring out which images were missing as they were not named after the actual shooting order. Luckily, the missing images were at the edges of the panorama so the consequences were really low. We cropped the zone on the left side of the panorama. Yes! We could have 2 more columns in the panorama because we used only 136 out of 138 shot columns => this would have been 27 Gigapixels.  Anyway…).

After spending an entire day finding and renumbering the missing images, we were able to use this new well-numbered set of images with the Clauss plugin of Autopano Giga in combination with the shooting log file.

The first optimization of the project with all images was quite good but not perfect because of some bad links remaining in the editor.  It’s easy to guess why: repeating patterns in an urban landscape are really common; for example, there is nothing more similar than two windows next to each other. This can lead to alignment issues in detected control points between neighboring images. That’s what we can see in the following screen shot which raises this issue :

This screen shot shows a standard detection without using the Clauss plugin of Autopano Giga. You can see a lot of red zones where all false links have been found. If you use a motorized head, the issue is not as big as this one. It will be really low but can still exist. Nevertheless, isn’t that screen shot beautiful ?

With practice, coping with these false links is quite quick. What takes much longer is coping with out-of-focus zones (for example with the top of the roof of St-Sulpice tower). No miracles here. You need to manually move around the images with the move mode in the editor. A full day of work was needed to do that because even if each operation was really easy to do, it took time for each move operation to happen on a panorama of that size. Regular saving of the project is highly advised.

Once we arrived at that step, we had a perfectly geometrical panorama. But we still need to cope with color correction. The lighting of the scene changed a lot during such a long shooting so we needed to be careful with that. BTW : yes, if you look accurately at the image, you can see that shadows moved in the picture. Not easy to guess, but it’s findable. So we put a lot of color anchors in the panorama :

With the help of multiple selection, it’s a fairly quick operation. Optimization was done right after so we have a nearly finished project, as illustrated below :

We still need to crop, but we’ve got the final size : 354159 x 75570 = 26,763,795,630 pixels. Wow. Not bad ;)

We can now proceed to the rendering.

Alexandre Jenny

Making of: step 1, the shooting

Thursday, March 11th, 2010

After 16 months of waiting on the weather, the D Day had finally arrived. All the conditions were in the right place at the right time on September 8th, 2009.
The Paris 26 Gigapixels project has 2 goals :

  1. Make the most beautiful gigapixel photo of Paris
  2. Create the biggest image ever stitched

8:00 AM Rallying the whole team at the bottom of the tower of Saint Sulpice. The day promises to be exceptional, everybody is crossing their fingers that the weather remains sunny throughout the day. The tower was being restored so Martin asked the foreman if we could use the crane to mount the equipment on top of the tower. Request granted, we were without our bags to begin the ascent of the tower.

9 :15 AM We set up the motorized panoramic head on the parapet of the tower. We Installed  2 Canon 5D Mark II (21.1 MP) on it, each with a 300 mm f4.0 with a tele converter in order to get a  600mm /f8.0 needed to beat the record.  We used 2 cameras simply to reduce the shooting time in half.

The cameras were set as follows:

  • 300mm f4.0 with a tele converter 2x (equivalent 600mm f8.0)
  • Manual focus. We used the Live View (zoom and pan) to get a very precise focus control.
  • Priority diaphragm F13 (to have a bigger depth of field)
  • Iso 800
  • Speed 1/800 in order to reduce the heat haze
  • Recording in RAW
  • Compact Flash 16GB
  • The motorized head triggers the cameras.

9: 30 Setting the panoramic head (horizontal field angle: 219.75 ° and 38.49 ° vertical). According to preliminary estimations with  a 30% overlapping between images we got:

  • 138 columns by 17 rows for a total of 2346 photos and for a shooting time around 2:30.

Unfortunately during the adjustment we found that one Canon takes blurred images.  Is this the fault of the lens or the tele converter? We didn’t have time to ask these questions. We must make a decision: cancel the shot or continue with one camera….

10:00 The decision was made to continue with one camera. Everyone was ready to spend 5 hours on the top of the tower. The shooting strategy was to take pictures horizontally (left to right) from the horizon and then down to the foot of the tower. At the beginning of each line we paused the panoramic head to manually adjust the  focus through the Live View  and change the memory card if necessary.

13:30 The team supplies (Alex & Martin) came back with sandwiches and some drinks. This break was welcome, however, Lionel and Arnaud  always kept an eye on the panoramic head because nothing could stop the shot.

14:45: The stress falls. We could finally breathe, it was finished.  We only needed to copy the memory card to  the laptop one last time.

15:15: The light was so exceptional that we decided to redo the first 2 lines of the shooting. This series of images may be used in assembly later (just in case).

16:50: The equipment was clustered in the middle of the tower. Martin signaled the crane operator and in no time the equipment had been laid at the bottom of the tower.

17:00 Everyone had the same question in mind: did we forget to shoot a photo ? The question will be answered in step 2 “The stitching”

Paris 26 Gigapixels as lived by its photographer Arnaud Frich

Thursday, March 11th, 2010

Trying to break a world record is never an easy task and obviously the project we got ourselves into wasn’t easy to carry out. We knew from the start that we would have to shoot over 2000 images. Despite this fact I quickly suggested working with RAW files because of the substantial improvement the format would bring to the quality of the final JPG images we would be using for the stitching. After a bit of hesitation, we decided to take this road. Working with RAW files allowed me, when processing the 2346 individual files, to play with various optimization parameters during that first step: the white balance, noise reduction, highlight management and harmonization of the various rows and columns of source images.  We knew that the actual shooting conditions were such that we would have to make some technical choices on location.

Since the shooting was taking place from a high point since we wanted to have a view of the whole city, I knew we would have one paradox to deal with: we needed a very nice sunny, clear day with good atmospheric transparency; but a sunny day means that the air would warm up. Even if we would be working very early in the morning before the sun would warm up the atmosphere too much, it would just be impossible to finish early because of the large amount of images we needed to shoot. With a 600 mm lens the atmospheric warm up would translate into turbulences that would invariably degrade image quality and sharpness when viewing them at 100%.   The difference switching from a 300mm to a 600mm is striking!  In order to keep this problem under control, the astronomers know that the shutter speed must be kept to the shortest possible amount. However, we needed to maintain an acceptable depth of field.  I then decided on the following compromise: 800 ISO, F13 with a 600mm lens to shoot by a bright sunny day using a shutter speed of 1/800sec. Even if I was going to be using a Canon 5D Mark II I knew I would end up with some noise in the final images.  I also knew that I would be able to remove it using RAW files and Photoshop’s Camera Raw.  The final result is quite acceptable!

Then, always looking to keep the noise under control, I worked with a shutter speed of 1/800sec in order to slightly overexpose my shots. A speed of 1/1600 at 800 ISO would have been OK in regards of the light conditions on the day the shooting took place, but using 1/800sec allowed me to slightly darken my images in Camera Raw and recover the highlights at the same time. Finally, overexposing allowed me to adjust my white balance without modifying my files and this would have become a lot more complex if I had been working with JPG files from the start. I suggested to my partners in this adventure to develop the image with a slight warm tone so we could reunite the pictures taken in the morning with the ones taken at the end of the afternoon that had a warmer tone. Harmonizing the 17 rows and the 138 columns of images was performed quite comfortably using groups of images in Camera Raw and then synchronizing them. The slight remaining differences were processed directly during the stitching process by Autopano Giga. I am still amazed by the quality of the final harmonization. Especially when you know that the first shot was taken at 11:30 am and the last at 04:30 pm!

Awaiting the image of Paris…

Wednesday, March 10th, 2010