[Vision] EXR Extended Dynamic Range Image File Format (feat. OpenEXR, 무손실 고해상도 데이터 저장)

💡 본 문서는 '[Vision] EXR Extended Dynamic Range Image File Format (feat. OpenEXR)'에 대해 정리해놓은 글입니다.
무손실 고해상도 데이터 저장을 위해 사용하는 포멧인 Extended Dynamic Range(EXR) Image File Format에 대한 내용을 정리하였으니 참고하시기 바랍니다.

1. What is EXR File format 

Overview of the OpenEXR File Format

Industrial Light + Magic implemented its own extended dynamic range file format in Summer 2000. The existing 8-bit file format used at the time could not accurately reproduce images with extreme contrast between the darkest and brightest regions, or images with very subtle color gradations.

ILM's extended dynamic range file format has been employed successfully in the movies Harry Potter, Men in Black II, and Signs. Several shows currently in production at ILM are using the new format.

Realizing that various other parties are interested in an extended dynamic range file format, ILM decided to polish its new file format a bit, and to publish it. OpenEXR is the result.

Features of OpenEXR

1) high dynamic range

Pixel data are stored as 16-bit or 32-bit floating-point numbers. With 16 bits, the representable dynamic range is significantly higher than the range of most image capture devices: 109 or 30 f-stops without loss of precision, and an additional 10 f-stops at the low end with some loss of precision. Most 8-bit file formats have around 7 to 10 stops.

2) good color resolution

with 16-bit floating-point numbers, color resolution is 1024 steps per f-stop, as opposed to somewhere around 20 to 70 steps per f-stop for most 8-bit file formats. Even after significant processing, for example extensive color correction, images tend to show no noticeable color banding.

3) compatible with graphics hardware

The 16-bit floating-point data format is fully compatible with the 16-bit frame-buffer data format used in some new graphics hardware. Images can be transferred back and forth between an OpenEXR file and a 16-bit floating-point frame buffer without losing data.

4) lossless data compression

The data compression methods currently implemented in OpenEXR are lossless; repeatedly compressing and uncompressing an image does not change the image data. With the current compression methods, photographic images with significant amounts of film grain tend to shrink to somewhere between 35 and 55 percent of their uncompressed size. New lossless and lossy compression schemes can be added in the future.

5) arbitrary image channels

OpenEXR images can contain an arbitrary number and combination of image channels, for example red, green, blue, and alpha, Y, U, and V (luminance, and two sub-sampled chroma channels), depth, surface normal directions, or motion vectors.

6) ability to store additional data

Often it is necessary to annotate images with additional data; for example, color timing information, process tracking data, or camera position and view direction. OpenEXR allows storing of an arbitrary number of extra "attributes", of arbitrary type, in an image file. Software that reads OpenEXR files ignores attributes it does not understand.

7) easy-to-use C++ and C programming interfaces

In order to make writing and reading OpenEXR files easy, the file format was designed together with a C++ programming interface. Two levels of access to image files are provided: a fully general interface for writing and reading files with arbitrary sets of image channels, and a specialized interface for the most common case (red, green, blue, and alpha channels, or some subset of those). Additionally, a C-callable version of the programming interface supports reading and writing OpenEXR files from programs written in C.

8) portability

The OpenEXR file format is hardware and operating system independent. While implementing the C and C++ programming interfaces, an effort was made to use only language features and library functions that comply with the C and C++ ISO standards. The resulting code is known to work on Linux (Intel x86), SGI Irix (Mips), and Mac OSX (PowerPC).

Example EXR File Header

iPhone Depth Image를 뽑는 과정에서 exr 형태 파일을 제공하여, 이를 분석하는 과정에서 Header를 추출한 내용이니 참고하시기 바랍니다. 해당 Header 정보의 경우 아래의 예시코드에 print(exrfile.header()) 만 추가하여 확인하였습니다.

{ 
    'channels': {'R': HALF (1,1)}, 
    'compression': ZIP_COMPRESSION, 
    'dataWindow': (0,0) - (191,255), 
    'displayWindow': (0,0) - (191,255), 
    'lineOrder': INCREASING_Y, 
    'pixelAspectRatio': 1.0, 
    'screenWindowCenter': (0.0,0.0), 
    'screenWindowWidth': 1.0
}

• channels: 'R'은 Red Intensity 이며, Depth 데이터를 저장하기 위해 하나의 채널을 사용한 것으로 보임
  • HALF: The IEEE-754 16-bit floating-point data type. Same as Nvidia’s 16-bit floating-point format (fp16 / half). The range of representable numbers is roughly 6.0×10^-8 ~ 6.5×10^4
• compression: 데이터 압축 방식
• dataWindow, displayWindow: dataWindow는 데이터 자체의 크기를 의미하며, displayWindow는 이를 이미지 상에 보여줄 때 사이즈
• lineOrder: 이미지 저장 방식으로, INCREASING_Y는 일반적으로 이미지 저장하는 방식과 동일하게 좌상단 (0,0)으로 시작
• pixelAspectRatio: Width divided by height of a pixel when the image is displayed with the correct aspect ratio. 
• screenWindowCenter, screenWindowWidth: Those programs(OpenEXR) should set screenWindowWidth to 1, and screenWindowCenter to (0, 0).
• [참고] OpenEXR File Header: https://openexr.com/en/latest/TechnicalIntroduction.html#header

2. How to Use an EXR File

How to Open an EXR File

1) Commercial Tools

• The fastest way to view an EXR image is online, using OpenHDR Viewer.
• Another free program that supports the format is the GIMP image editor.
• Other ways include using Adobe Photoshop or Adobe After Effects.
• ColorStrokes and advanced imaging programs should also be able to open EXR files.

2) using OpenEXR

def read_depth_exr_file(filepath):
    # exrfile = exr.InputFile(filepath.as_posix())
    exrfile = exr.InputFile(filename)
    raw_bytes = exrfile.channel('R', Imath.PixelType(Imath.PixelType.FLOAT))
    depth_vector = np.frombuffer(raw_bytes, dtype=np.float32)
    height = exrfile.header()['displayWindow'].max.y + 1 - exrfile.header()['displayWindow'].min.y
    width = exrfile.header()['displayWindow'].max.x + 1 - exrfile.header()['displayWindow'].min.x
    depth_map = np.reshape(depth_vector, (height, width))
    return depth_map

import numpy as np
import OpenEXR as exr
import Imath

from matplotlib import pyplot as plt

def readEXR(filename):
    """Read color + depth data from EXR image file.
    
    Parameters
    ----------
    filename : str
        File path.
        
    Returns
    -------
    img : RGB or RGBA image in float32 format. Each color channel
          lies within the interval [0, 1].
          Color conversion from linear RGB to standard RGB is performed
          internally. See https://en.wikipedia.org/wiki/SRGB#The_forward_transformation_(CIE_XYZ_to_sRGB)
          for more information.
          
    Z : Depth buffer in float32 format or None if the EXR file has no Z channel.
    """
    
    exrfile = exr.InputFile(filename)
    header = exrfile.header()
    
    dw = header['dataWindow']
    isize = (dw.max.y - dw.min.y + 1, dw.max.x - dw.min.x + 1)
    
    channelData = dict()
    
    # convert all channels in the image to numpy arrays
    for c in header['channels']:
        C = exrfile.channel(c, Imath.PixelType(Imath.PixelType.FLOAT))
        C = np.fromstring(C, dtype=np.float32)
        C = np.reshape(C, isize)
        
        channelData[c] = C

    # Read EXR RGBD Format
    colorChannels = ['R', 'G', 'B', 'A'] if 'A' in header['channels'] else ['R', 'G', 'B']
    img = np.concatenate([channelData[c][...,np.newaxis] for c in colorChannels], axis=2)
    
    # linear to standard RGB
    img[..., :3] = np.where(img[..., :3] <= 0.0031308,
                            12.92 * img[..., :3],
                            1.055 * np.power(img[..., :3], 1 / 2.4) - 0.055)
    
    # sanitize image to be in range [0, 1]
    img = np.where(img < 0.0, 0.0, np.where(img > 1.0, 1, img))
    
    # Read EXR D Format
    Z = None if 'Z' not in header['channels'] else channelData['Z']
    
    return img, Z

if __name__ == "__main__":
    filename = "./nerf_custom/lab_record3d/EXR_RGBD/depth/0.exr"
    img, Z = readEXR(filename)
    Z = readEXR(filename)

    plt.imshow(Z)
    plt.imsave("depth.jpg", Z)

depth_read_exr.py

0.00MB

How to Convert an EXR File

AConvert.com is an example of an online tool that supports this format. It can save your file to JPG, PNG, TIFF, GIF, and many other formats. This website can also resize it before the conversion.

You might also be able to convert an EXR file using one of the programs from above that can open the file, but a dedicated file converter is much quicker and doesn't need to be installed to your computer before you use it.

참고

• [OpenEXR] OpenEXR Official Docs: https://openexr.com/en/latest/index.html#openexr
• [OpenEXR] OpenEXR Technical Docs: https://openexr.com/en/latest/TechnicalIntroduction.html
• [massive] What is an EXR File?: https://massive.io/file-transfer/what-is-an-exr-file/
• [Blog] What Is an EXR File?: https://www.lifewire.com/exr-file-2621157
• [Github Issue] How to read the depth map?: https://github.com/bmild/nerf/issues/77
• [stackoverflow] Reading .exr files in OpenCV: https://stackoverflow.com/questions/44630793/reading-exr-files-in-opencv
• (beginner) EXR /w Depth map - Compression:  https://www.steakunderwater.com/wesuckless/viewtopic.php?t=5589