Oracle today open-sourced Graphpipe, a tool created to make it easy to serve machine learning models in the cloud. Graphpipe was designed to simplify the deployment of machine learning for use on mobile apps and IoT devices, as well as web services for end users or AI for internal use at companies.
Oracle today open-sourced Graphpipe, a tool created to make it easy to serve machine learning models in the cloud made by popular frameworks like TensorFlow, MXNet, Caffe2, and PyTorch. Graphpipe was designed to simplify the deployment of machine learning for use on mobile apps and IoT devices, as well as web services for end users or AI for internal use at companies.
“Graphpipe is an attempt to standardize the protocol by which you speak to a remotely deployed machine learning model, and it includes some reference servers that allow you to deploy machine learning models from existing frameworks very easily in an efficient way,” Oracle cloud architect Vish Abrams told VentureBeat in a phone interview. Prior to joining Oracle, Abrams led efforts at NASA to open-source the OpenStack cloud computing platform.
Use of the tool may mean developers don’t have to build custom APIs to deploy AI models or be concerned about which popular framework was used to create a model.
The tool is available for free on Github and joins a series of open source tools launched in recent years for developers who want to use AI, including popular frameworks like TensorFlow. The Open Neural Network Exchange (ONNX) was created roughly a year ago by Facebook and Microsoft and proposes standard formats for machine learning models to allow interoperability between frameworks.
Developers today have a lot of options when it comes to frameworks for creating AI models, Abrams said, but fewer for how to serve or deploy AI models.
“It seems like people haven’t really thought about this too much, and my suspicion is that it’s a case where machine learning has grown so dramatically in the past few years that everybody is still in the experimental R&D phase and building their models. They’re not really thinking about ‘OK, how do I take this and put it into production somewhere?,’” Abrams said. “So the tooling around the deployment side is just kind of nascent, but it’s going to be much stronger over the next few years, so hopefully this protocol is kind of like the first step to creating some really robust tooling around model deployments.”
Oracle created and open-sourced Graphpipe to provide a service to the wider AI ecosystem, Abrams said, a chance for Oracle to “develop and improve really modern technology and not just be sort of a company of the past.”
The Graphpipe network protocol for transmitting components in a deep learning architecture includes guidelines for serving AI models, examples for serving models, and client libraries for querying models Graphpipe serves up.
