Spark Operator Demo

by jkremser 5 years ago

GNU/Linux ◆ xterm-256color ◆ bash 419 views

Repository

Commands used in the recording

creates the operator

kubectl apply -f manifest/operator.yaml

creates a new Spark cluster w/ two workers

kubectl apply -f examples/cluster.yaml

scales down the cluster by editing the Custom Resource representing the cluster

kubectl edit sparkcluster my-spark-cluster

deleting the Spark cluster

kubectl delete sparkcluster cm my-spark-cluster

print another cluster definition cat examples/with-prepared-data.yaml result:

apiVersion: v1
kind: ConfigMap
metadata:
  name: spark-cluster-with-data
  labels:
    radanalytics.io/kind: cluster
data:
  config: |-
    workerNodes: "2"
    masterNodes: "1"
    downloadData:
    - url: https://data.cityofnewyork.us/api/views/kku6-nxdu/rows.csv
      to: /tmp/
    - url: https://data.lacity.org/api/views/nxs9-385f/rows.csv
      to: /tmp/LA.csv

expose NodePort

kubectl expose rc my-spark-cluster-m --type=NodePort

get the spark master url

kubectl get services
minikube service my-spark-cluster-m --url

attach with spark-shell

spark-shell --master=spark://192.168.39.87:32371

More by jkremser

untitled 00:39

by jkremser 1 month ago

untitled 06:42

by jkremser 5 months ago

k luster create 01:08

by jkremser 4 months ago

untitled 01:57

by jkremser 5 months ago

See all

You can download this recording in asciicast v2 format, as a .cast file.

Download

Replay in terminal

You can replay the downloaded recording in your terminal using the asciinema play command:

asciinema play 189204.cast

If you don't have asciinema CLI installed then see installation instructions.

Use with stand-alone player on your website

Download asciinema player from the releases page (you only need .js and .css file), then use it like this:

<!DOCTYPE html>
<html>
<head>
  <link rel="stylesheet" type="text/css" href="asciinema-player.css" />
</head>
<body>
  <div id="player"></div>
  <script src="asciinema-player.min.js"></script>
  <script>
    AsciinemaPlayer.create(
      '/assets/189204.cast',
      document.getElementById('player'),
      { cols: 173, rows: 43 }
    );
  </script>
</body>
</html>

See asciinema player quick-start guide for full usage instructions.

While this site doesn't offer GIF conversion at the moment, you can still do it yourself with the help of asciinema GIF generator utility - agg.

Once you have it installed run the following command to create GIF file:

agg https://asciinema.org/a/189204 189204.gif

Or, if you already downloaded the recording file:

agg 189204.cast 189204.gif

Check agg --help for all available options. You can change font family and size, select color theme, adjust speed and more.

See agg manual for full usage instructions.