Image Reshaping for Efficient Compression of Plenoptic Content

Abstract-With insertion of an additional microlens array in front of the imaging sensor, plenoptic camera, which acquires light field by a single exposure, attracts great interests both from academy and industry. A plenoptic image records both spatial and angular light radiance in a super high definition with macropixel structures, which results in a distinct intensity distribution. Several compression techniques have been proposed, however, most of them are suffered from the standard incompatibility and high compression complexity. In this paper, a novel and standard compatible plenoptic data reshaping algorithm is proposed to align the macropixel structures in the lenslet image with the coding unit grids of block-based image/video coding standards before compression. After encoding and decoding the reshaped lenslet image using the existing block-based image/video coding standards, the inverse image reshaping is used to convert the reconstructed image back to the original light field data format for further applications. The experimental results reveal that the proposed method introduces 10.47% and 27.64% bitrate reduction to HEVC/H.265 Format Range Extension (RExt) Profile and AVC/H.264 High 4:4:4 Profile, respectively, by compressing the reshaped lenslet images. Also, it brings additional compression efficiency improvements to intra prediction tools, such as Intra Block Copy (IBC) and Locally Linear Embedding (LLE) based coding method, by an average of 11.62% and 9.81% bitrate reduction as they compress the reshaped plenoptic images.