(This is my very first post here, so I’m sure I violate a few rules of this community, mainly by asking too much questions in a simple post. Sorry for that.)
I have a monolithic ASP.NET Core MVC application which I develop as a hobby project for private use. It can download videos and music from several video sharing websites by using youtube-dl. This application simply shell executes youtube-dl using the correct cmdline arguments, captures stdout and stderr, does some parsing on it, and reports any status changes real-time to the frontend using SignalR. Also, every day when it is idle, shell executes youtube-dl to update itself. I made some changes in the code to limit the maximum parallel downloads to a specified value in appsettings.json.
It does its job fine on a single server. But at work I became an architect associate, learned about Kubernetes, and from scratch I never made my own software for Kubernetes. So, I want to make the following changes:
- Refactor the code from monolithic to microservice architecture…
- …by considering Kubernetes as the target platform.
Question 0: Does it make sense to make these changes and migrate this project to Kubernetes, considering how youtube-dl is used?
On the backend side, I have two tasks, and both of them requires youtube-dl.
- By using the URI given by the user, I have to collect information about the media (title, description, thumbnail image). This task is only network-heavy until youtube-dl gathers the media information and it is done within a few seconds.
- Also by the given URI, I have to download the media and present a file to the user. Also, I need to keep the real-time progress report. This is both network and CPU-heavy, since youtube-dl not only downloads the media, but alsp performs video merging and video to audio conversion using FFmpeg. So it takes some time and CPU resources.
Question 1: Sould I separate these responsibilities to two different backend microservice? If I have it separated, I would be able to keep only a few instances for the first task, and several instances for the second (splitted to different nodes). On the other hand, I need to use youtube-dl for both the tasks, so it would also make sense to put it in one service.
My next consideration is the real-time process feedback to the frontend. By separating this project, I have to make a chain: DownloaderMS->FrontendMS->HTML frontend. In the old world I used SignalR. According to the Microsoft docs, using SignalR between DownloaderMS and Frontend does not violate the microservice architecture until I use Redis backplane.
Question 3: By considering Kubernetes and load balancing, is it a good idea to keep using SignalR both on frontend and backend side? Or should I use something else?
The software needs some storage space to store the downloaded media. I think I should have a persistent volume with read-write access for the backend services with 1 hour data retention.
Question 4: Should the backend microservices encapsulate the persistent volume and provide the downloaded media for the FrontendMS on an API endpoint, or is it allowed in microservice architecture to let the FrontendMS access this volume but with read-only permissions?
Question 5: Do you have any other considerations or recommendations?