big sur – Reinstall python 2.7.16 librairies on MacOS Big Sur 11.3

To confirm results of a colleague who has coded an interpolation with Cubicspline from scipy with python2 , I have to import CubicSpline function. I know python2 is deprecated but this is just for a validation.

But when i do a :

from scipy.interpolate import CubicSpline

I have systematically the error message :

ImportError: cannot import name CubicSpline

So, I tried to install it with pip-2.7 by doing :

curl https://bootstrap.pypa.io/pip/2.7/get-pip.py -o get-pip.py

python2.7 get-pip.py

and after from the directory ~/Library/Python/2.7/bin :

./pip2.7 install scipy

but same error, CubicSpline is not found

I have the version :

$ which python2.7
/usr/bin/python2.7

So python2.7 is the default python2 of the system.

How to install CubicSpline library on the default installation of python2.7 in the OS ?

Have I got to reinstall all the python2.7 framework ? and if yes, how to perform it ?

python – Best way to migrate to new versions of librairies with breaking changes

(I orignally posted my question on StackOverflow and figured maybe this would be a better place to ask)

I provide details on the librairies I’m working with because it helps understand my issue but it could be applied to other projects as this is a general question on how to do a migration while still be able to use previously saved data.

We are currently using Tensorflow 1.15 for some MLP models and Stable Baselines (SB2) for our Reinforcement Learning models. I have the task to migrate the code to Tensorflow 2 and, because Stable baselines use Tensorflow 1.x as its backend but is not compatible with Tensorflow 2 (and won’t be), to also migrate it to Stable Baselines3 (SB3) which use PyTorch has its backend.

My issue is the following: we want to still be able to use our old trained models which are used in production (both MLP and RL ones) until they are not relevant anymore while at the same time starting to train new models on Tensorflow2 and SB3. This means keeping TF1 to be able to use SB2 and our previous trained model but also adding TF2 and SB2 support for future use.

Having both TF1 and TF2 at the same time might cause issues (it doesn’t seems to be possible in Python very easily unless I want to mess with installed packages).

What is the best way to do this ?

  • Should I migrate by developping same functionnality in newer versions (TF2 and SB3) and offer the choice to use both previous and new while warning about deprecation for TF1 and SB2?
  • Another solution might be to develop a new branch with TF2 and SB3 support only. Then run both service at the same time (each handling models for its corresponding version) until at some point I stop running the old one. It might be tricky to maintain both at the same time though, any thoughts on that ?

Thanks in advance for any help,

P.S. I don’t know if this is the right StackExchange forum for this kind of question so let me know if not 😉

EDIT More details

For now, clients all access the same site which is hosted on a single machine. They can train, define and use models. There are 3 main services which are docker images running as service on Linux host. One service is in charge of training and evaluating models, one service for making prediction continuously with said models and finally a service that use results to control some other software/hardware. Thus they all use SB2 and Tensorflow 1.x models for now.

What are the best pratices, frameworks, librairies to integrate Java plugins?

OK! Lets first state that I am a senior software developer, but it makes a little while I did not start from scratch a new software. I already looked around, but I would like to have your opinions and maybe a discussion. Thank you in advance!

Requirements:

  • Shall be using plugins
  • Plugins may or may not have dependencies (maybe even between plugins, but I am not sure at this point)
  • The software will be able to support three starting modes (so only one at a time):
    • Command line
    • Web Server (which will include the frontend)
    • A window version
  • This will be an opensource project

Current development:

What I already have is something really basic. It is currently divided into two projects:

  • SoftwareBase: It is giving currently the possibility to start with two modes: Command line OR Window mode. The goal of this would be to not have to create a new base each time.
  • The software itself: That depends on the SoftwareBase one.

Within each projects, there are currently a package that includes all the commands (one per class) that can be executed and shall support the three modes (or maybe other classes would transpose their return (when there is) to the specific output (window, web, command line)).

So any new plugins could import their own commands. I already think about external or internal commands that, respectively, could be called by command line / programmatically or could only be called via a line of code.

Questions:

  1. I was asking myself if the web mode support shall be integrated as a plugin or being native.
  2. I already looked at dynamic JAR loading, but shall I implement this or is there any thing I could use already built? (I know there is, but remember I am reaching for your opinion)
  3. What do you think in general of this?

Thank you again for your guidance!