Write once, deploy anywhere
Sponsored Artificial Intelligence techniques have been finding their way into business applications for some time now. From chatbots forming the first line of engagement in customer services, to image recognition systems that can identify defects in products before they reach the end of the production line in a factory. But many organisations are still stuck at where to start in building machine-learning and deep-learning models and taking them all the way from development through to deployment. Another complication is how to deploy a model onto a different system than the one that was used to train it. Especially for situations such as edge deployments, where less compute power is available than in a datacentre. One solution to these problems is to employ OpenVINO™ (Open Visual Inference & Neural Network Optimization), a toolkit developed by Intel to speed the development of applications involving high-performance computer vision and deep-learning inferencing, among other use cases. OpenVINO takes a trained model, and optimises it to operate on a variety of Intel hardware, including CPUs, GPUs, Intel® Movidius™ Vision Processing Unit (VPU), FPGAs, or the Intel® Gaussian & Neural Accelerator (Intel® GNA). This means that it acts like an abstraction layer between the application code and the hardware. It can also fine tune the model for the platform the customer wants to use, claims Zoë Cayetano, Product Manager for Artificial Intelligence & Deep Learning at Intel. “That’s really useful when you’re taking an AI application into production. There’s a variety of different niche challenges in inferencing that we’ve tackled with OpenVINO, that are different from when models and applications are in the training phase,” she says. For example, Intel has found there is often a sharp decline in accuracy and performance when models that were trained in the cloud or in a datacentre are deployed into a production environment, especially in an edge scenario. This is because the trained models are shoe-horned into a deployment without considering the system they are running on. In addition, the model may have been trained in ideal circumstances that differ from the actual deployment environment. As an example, Cayetano cites a defect detection scenario in which a camera view close to the production line may have been assumed during training. The camera may be positioned further away so the images may have to be adjusted — which OpenVINO can do. Interoperability is another issue, according to Cayetano, “We were seeing this trend of a lot of businesses adopting AI and wanting to take AI into production.
