OpenShift Commons Big Data SIG #2: Running Apache Spark Natively on Kubernetes with OpenShift

In this Video

In this second meeting of the Big Data Special Interest Group, our guest speaker was Erik Erlandson, Senior Software Engineer at Red Hat on the Big Data team. Erik gave a brief introduction to RDD (Resilient Distributed Dataset) in the context of Apache Spark and then dove right into a demo of running Apache Spark natively on OpenShift, Red Hat’s Enterprise-ready Kubernetes for developers.

Apache Spark and Resilient Distributed Datasets
Apache Spark and Resilient Distributed Datasets

Synopsis

Apache Spark can be made natively aware of Kubernetes by implementing a Spark scheduler back-end that can run Spark application Drivers and bare Executors in kubernetes pods. In this talk, Erik will explain the design of a native-Kubernetes scheduler back-end in Spark and demonstrate a Spark application submission with OpenShift.

Oshinko on OpenShift
Oshinko on OpenShift

Also in the meeting, Erik mentioned yet another related project, Oshinko a user interface designed to help manage and deploy your Spark Cluster on OpenShif. Here’s a link to the video on that project which will be the subject of an upcoming BIG Data SIG in December!

 

If you’d like to join the Big Data SIG or find out more about OpenShift Commons, please visit https://commons.openshift.org

OpenShift Commons Gathering in Seattle November 7, 2016

The Big Data SIG will be meeting in person at the upcoming OpenShift Commons Gathering colocated with CNCF.io’s Kubecon. We will have a stellar lineup of speakers from across the OpenShift ecosystem and plenty of time set aside to network and meet with your peers. Find out more here: https://commons.openshift.org/gathering

Big Data SIG Playlist

You can find a Playlist of all previously recorded SIG sessions on YouTube.

Don’t forget to leave your feedback and suggestions for each video or in the comments section below. This will be incredibly important to shape this Special Interest Group and create sessions that fit the demands of all the OpenShift developers in the community.

Connect with the Big Data SIG and other OpenShift Commons members at OpenShift Commons Gathering

oscg-postcard-new

Both the Big Data SIG’s Chair Will Benton and Erik Erlandson will be participating at the upcoming OpenShift Commons Gathering along with 200+ other OpenShift community members. Join us and connect directly with your peers across the OpenShift and Kubernetes ecosystem in person.

In addition, all of the OpenShift Commons Special Interest Groups (SIGs) will be meeting face-to-face in Seattle at the upcoming OpenShift Commons Gathering which is conveniently co-located with Kubecon and CloudNativeCon at the Seattle Sheraton.

About OpenShift Commons Gathering Seattle November 7, 2016

All the OpenShift Commons members and many of the guests from past OpenShift Commons Briefings will be connecting in person at the upcoming OpenShift Commons Gathering co-located with CNCF.io’s Kubecon. We will have a stellar lineup of speakers from across the OpenShift ecosystem and plenty of time set aside to network and meet with your peers.

Find out more here: https://commons.openshift.org/gathering

OpenShift Commons Briefings Playlist

You can find a playlist of all previously recorded SIG meetings and OpenShift Commons Briefings on YouTube.

Don’t forget to leave your feedback and suggestions for each video or in the comments section below. This will be incredibly important to shape this Special Interest Group and create sessions that fit the demands of all the OpenShift developers in the community.

About OpenShift Commons

OpenShift Commons is the place for organizations that are part of the OpenShift community to connect with peers and other related open source technology communities to communicate and collaborate across all OpenShift projects and stakeholders.

The Commons’ goal is to foster collaboration and communication between OpenShift stakeholders to drive success for all members, and expand & facilitate points of connection between members for sharing knowledge and experience to help drive success for the platform and for participants: customers, users, partners, and contributors.

Join OpenShift Commons today http://commons.openshift.org/#join

Categories
Events, OpenShift Ecosystem, Thought Leadership, Videos
Tags
, , , ,