Pyspark brings together the analytical power and popularity of Python with the distributed-computing capability of Spark. In this post I show how you can use a docker container with pyspark and spark pre-loaded to let you play with pyspark in a Jupyter notebook, rather than having to configure your own spark cluster first. Use Jupyter… Read More »
This post shows a simple NiFi dataflow and tries to demonstrate some of the elements of creating a NiFi workflow including adding and connecting processors, attributes and properties. This simple NiFi dataflow takes files from an input directory and routes them to different folders depending on the file name. Configure NiFi and Docker Compose In… Read More »
NiFi is a tool designed to support the flow of data between software systems. This post shows how you can achieve NiFi version control with NiFi Register and docker. A note of caution: this post is about showing a little of what of possible with NiFi version control, and I suggest you read the NiFi… Read More »
JSON is a common format for sending data to and from a RESTful API. In this post we’ll see how we can allow a simple Flask API to receive a JSON input. This post extends my previous post which made a simple Flask RESTful API and uses the Flask ‘get_json()’ method to accept JSON input.… Read More »
Heroku is a ‘platform as a service’ (PaaS) cloud provider. Heroku makes it easy to deploy apps by pushing to git repositories or using docker containers. This post shows you how you can deploy docker to Heroku. The Heroku website already has some good guides to help you deploy docker to Heroku, but I found… Read More »