Yahoo, model Apache Spark citizen and developer of CaffeOnSpark, which made it easier for developers building deep learning models in Caffe to scale with..
Yahoo, model Apache Spark citizen and developer of CaffeOnSpark, which made it easier for developers building deep learning models in Caffe to scale with parallel processing, is open sourcing a new project called TensorFlowOnSpark. The pairing of Spark and TensorFlow should make the deep learning framework more attractive to developers who are creating models that need to run on large computing clusters.
For those that zoned out during the big-data boom, Apache Spark is an open source framework designed to increase the efficiency of parallel computing. Following in the steps of tools like Hadoop, Spark made it possible for companies like Netflix to process huge amounts of user data to offer up recommendations at scale.
Machine learning frameworks like Google’s TensorFlow and Caffe help people create deep learning models without the rigorous skill-set of a machine learning specialist. The open source libraries exist at a higher level of abstraction that enable developers to create models without getting lost in the weeds reinventing the wheel.
