Building a Microservice Based Big Data PipelineLaajuus (3 ECTS)

Objective

This practical, hands-on course will introduce you to the world of big data. The idea is to create a complete pipeline. Starting with gathering the data and ending with simple data analysis and visualizations. The components will be set-up in a microservice based architecture.

At the beginning of each session we will first see some theoretical background information. We will always keep a practical approach in mind. You will learn new technology and software concepts. Critical questions will challenge you and give you new insights.

Content

Linux, daily use:
- Introduction to Ubuntu Server, the command line interface (CLI)
- Shortcuts and tools, the ones you really should know about!
- Basic text-processing tools

Containerization with Docker:
- Containers & Docker Containers
- Microservices vs. Monoliths
- Virtualization vs. Containerization
- Docker eco-system
- Best-practices

Message queue – Kafka:
- What is a message queue?
- Kafka key concepts
- Microservice example: image recognition

Elastic Stack:
- Components, quick introduction
- Working with Elasticsearch and Kibana
- Simple text-analysis
- Simple visualizations

Qualifications

Basic Python programming skills
Linux (Ubuntu) knowledge is a plus

Assessment criteria, satisfactory (1)

Attending all the labs, completing all the assignments in-time. Able to explain the created pipeline.

Assessment criteria, good (3)

Attending all the labs, completing all the assignments in-time. Able to explain the created pipeline.
Completed some of the extra exercise. 

Assessment criteria, excellent (5)

Attending all the labs, completing all the assignments in-time. Able to explain the created pipeline.
Completed all the extra exercises. 

Assessment criteria, approved/failed

Active attendance on all lab sessions.
Completing practical exercises.