Sony Corporation and Sony Semiconductor Solutions Corporation have released world’s first image sensor equipped with an Artificial Intelligence (AI) processing function. There are two models, the bare chip product “IMX500” and the package product “IMX501”.
Sony IMX500 and IMX501 specifications include sensor with 12.3 effective mega pixels, the sensor size is 7.857mm diagonal (1 / 2.3 type), the pixel size is 1.55μm, the frame rate is 60fps at full resolution, 4K (4,056). X2,288 pixels) 60 fps, 1080p 240 fps, Full / Video + AI processing and metadata output 30 fps.
The sensitivity is about 250 LSB at the standard value F5.6, and the sensor saturation signal amount (minimum value) is about 9,610 e-. In addition, it also has ISP and HDR shooting functions.
It is expected to be used in applications such as the retail industry and the industrial equipment industry by developing cameras equipped with AI functions that use this product.
In recent years, various devices are connected to the cloud due to the spread of IoT, and it is common to perform AI processing on the information acquired from devices on the cloud, but on the other hand, the amount of information handled by the cloud side increases. There are issues such as data transfer delays and security concerns caused by passing through the cloud.
This product uses a layered structure in which a pixel chip and a logic chip equipped with an image analysis processing function by AI are stacked, and it is possible to perform AI processing in the sensor on the signal acquired by the pixel chip without going through the cloud. It also has the advantage of not requiring a high-performance processor or external memory.
In addition, by processing in the chip, it is possible to output only metadata that does not output image information, reducing the amount of data and protecting privacy, real-time tracking of objects by high-speed AI processing, rewriting of internal memory.
With various functions such as selecting an AI model according to the user’s usage environment and conditions, it can be used in various applications. For example, at stores, it can be used for counting visitors, detecting out-of-stock items on shelves, and heat maps of customers (detection of places where many people gather).