version control – Continuous Integration practices for forked repository

We have an on-premises BitBucket server with a git repo for an embedded device. We use a multi-branch pipeline in Jenkins to:

  • Build and run tests
  • Build the firmware (and a little supporting PC tool)
  • (In the near future) run hardware-in-the-loop integration tests
  • Upload artifacts to Artifactory (if the branch name meets certain requirements)

Now, for a spin-off project which uses the same hardware, I have created a fork of the repository in BitBucket, of which I start to doubt the added value now (elaboration at the end of my post). I will be the main contributor of this, just like in the other repo.

To have good CI practices in the spin-off too, I cloned the VM which contains the Jenkins instance and changed its pipeline to use the fork. To not have duplicated build time of the main branches of the original project, I configured some branch name filters. I created a secondary develop branch (let’s call it spinoff-develop) in the fork and set this as default, to allow integrating the spin-off features together, while keeping develop clean, allowing receiving upstream updates (which are automatically synced from the original by BitBucket, this would otherwise break). So far so good, the Jenkins clone now runs builds for any branch not named master or develop.

Now I run into the issue that there is some logic in my Jenkinsfile to determine when to upload a build to Artifactory. Of course I can change that logic in the fork, but when I eventually decide to merge useful changes from the fork into the original project, those will be changes which I can’t accept there. Should I start playing around with Jenkins environment variables to get around this? I don’t prefer that, as I want to keep as little configuration as possible in Jenkins, because I regard it “volatile” and it should not take too much time to set it up again if anything bad happens to it (there are also initiatives in the company to automate this more with a kind of “Infrastructure as code” approach but I don’t use them yet).

As mentioned above, I doubt the direction where I am going with this, because of the increased complexity. The reason for the fork is that I am not 100% certain yet whether the spin-off contributions will be eventually merged back into the original project, but I want easy integration of upstream changes into the spin-off. Also, if another team would ever take over the spin-off, it’s easy to just grant them full access to the fork in BitBucket and give them the VM.

What are your thoughts about this?

EDIT: I could add Jenkinsfile-spinoff and have the clone use that. But my main concerns do still apply, and is this a good solution in the end? Suppose there were 100 forks, would there need to be 101 Jenkinsfiles?

numerics – Speed up Multidimensional numerical integration

I have the following 9-D integral, where the domain of integration is a nine-dimensional hypercube of unit length. The problem is that Mathematica takes too long to evaluate this. Is there any specific way to speed it up using parallelization or using specific integration strategy?

  u1 = 1; u2 = 10^(12); u3 = 
         10^(-12); u4 = 1; u5 = 1; u6 = 100; u7 = 0.01; u8 = 10000; u9 = 
         10^(-8);

    (Alpha) = 0.3; (Beta)1 = 0.25; (Beta)2 = 0.4; (Beta)3 = 0.54; 
    (Gamma)1 = 0.6; (Gamma)2 = 0.7;

w1 = u3; w2 = u3 u5; w3 = u3 u5 u7; w4 = u9; w5 = u2 u3 u9; w6 = 
 u2 u3 u4 u5 u9; w7 = u8 u9; w8 = u1 u2 u3 u8 u9; w9 = u6 u8 u9; 

num = 1/(2 (Pi) I)^9 (w1)^z1 (w2)^z2 (w3)^z3 (w4)^z4 (w5)^z5 (w6)^
   z6 (w7)^z7 (w8)^z8 (w9)^
   z9 Gamma(-z1) Gamma(-z2) Gamma(-z3) Gamma(-z4) Gamma(-z5) 
Gamma(-z6) Gamma(-z7) Gamma(-z8) Gamma(-z9) Gamma((Alpha) + z1 + z2 +
      z3 + z4 + z5 + z6 + z7 + z8 + z9) Gamma((Beta)1 + z1 + z5 + 
     z8 + z9) Gamma((Beta)2 + z2 + z6 + z7 + z8) Gamma((Beta)3 + 
     z3 + z4 + z5 + z6) Gamma(
    1 - (Gamma)2 - z4 - z5 - z6 - z7 - z8 - z9) Gamma(
    1 - (Gamma)1 - z1 - z2 - z3 - z5 - z6 - z8);

limit = 1;

num1 = num /. {z1 -> -0.04 - limit I + 2 I z1, 
     z2 -> -0.02 - limit I + 2 I z2, z3 -> -0.01 - limit I + 2 I z3, 
     z4 -> -0.005 - limit I + 2 I z4, 
     z5 -> -0.0025 - limit I + 2 I z5, 
     z6 -> -0.00125 - limit I + 2 I z6, 
     z7 -> -0.000625 - limit I + 2 I z7, 
     z8 -> -0.0103125 - limit I + 2 I z8, 
     z9 -> -0.0101563 - limit I + 2 I z9} // Simplify;

(2 I)^9 NIntegrate(
   num1, {z1, 0, limit}, {z2, 0, limit}, {z3, 0, limit}, {z4, 0, 
    limit}, {z5, 0, limit}, {z6, 0, limit}, {z7, 0, limit}, {z8, 0, 
    limit}, {z9, 0, limit}) // AbsoluteTiming

numerical integration – Different Methods in NIntegrate

I am trying to numerically calculate a multidimensional integral which involves Jacobi elliptic theta functions. The integrand is the following:

integrand(d_, x_, y_, xp_, x0_, T_) := 
 T^(-(d + 1)/2) (d-1 - y^2/(4T)) Exp(-y^2/(8T)) 
( EllipticTheta(3, 1/2 Pi (-x-x0), Exp(-Pi^2 T) ) + EllipticTheta(3, 1/2 Pi (-x+x0), Exp(-Pi^2 T) ) ) 
( EllipticTheta(3, 1/2 Pi (-xp-x0), Exp(-Pi^2 T) ) + EllipticTheta(3, 1/2 Pi (-xp+x0), Exp(-Pi^2 T) ))

My goal is to integrate this expression with respect to x0 and T for d=3, and get a 3d plot of the result as a function of x and xp (both variables between 0 and 1) – while manipulating y. After this, I need to take the derivative of the integrated result with respect to both x and xp.

For the integration, I have tried 3 different strategies. In the first one, I do not specify the Method:

integral(d_?NumericQ, x_?NumericQ, y_?NumericQ, xp_?NumericQ) := 
 NIntegrate(
  integrand(d, x, y, xp, x0, T), {T, 0, ∞}, {x0, 0, 1}, 
  PrecisionGoal -> 10, MinRecursion -> 10, MaxRecursion -> 20)

I’ve found that increasing the MinRecursion changes the results, and 10 seems to work well (higher values do not seem to improve the results). Since generating the full 3D plot takes somewhat long, I then generated the following table:

Table(integral(3, x, 1, 0), {x, 0.05, 1, 0.05})

with the outcome

{-43.386, -38.7746, -34.1253, -31.4359, -26.9778, -22.7969, -19.8602, -20.2972, -13.8984, -6.49645, -3.3476, -3.31147, 6.20662, 8.2472, 12.0905, 13.7228, 14.896, 15.814, 16.3162, 16.463}

In a second attempt, I tried Method->"LocalAdaptive" for the integration:

adaptintegral(d_?NumericQ, x_?NumericQ, y_?NumericQ, xp_?NumericQ) := 
 NIntegrate(
  integrand(d, x, y, xp, x0, T), {T, 0, ∞}, {x0, 0, 1}, 
  PrecisionGoal -> 10, MinRecursion -> 10, MaxRecursion -> 20, 
  Method -> "LocalAdaptive")

which produces the following numbers for the same table:

{-20.7877, -19.7131, -17.9935, -15.7272, -13.0363, -10.0544, -6.91493, -3.74124, -0.63984, 2.30356, 5.02495, 7.48073, 9.64493, 11.5056, 13.061, 14.316, 15.2788, 15.9584, 16.3626, 16.4967}

The outcome is very different compared to the first table, and since I did not get any error messages, I wonder if there is a way to tell which gives a more accurate estimate of the actual result.

I also tried the Method->"MonteCarlo":

mcintegral(d_?NumericQ, x_?NumericQ, y_?NumericQ, xp_?NumericQ) := 
 NIntegrate(
  integrand(d, x, y, xp, x0, T), {T, 0, ∞}, {x0, 0, 1}, 
  PrecisionGoal -> 10, MinRecursion -> 10, MaxRecursion -> 20, 
  Method -> "MonteCarlo")

which gives the following values for the same table

{-21.2913, -19.2249, -18.663, -16.2671, -13.3218, -9.81518, -4.44489, -3.11635, -0.264413, 2.72884, 4.44556, 8.09827, 9.49501, 11.4452, 13.0165, 14.0828, 15.279, 16.3008, 16.6255, 16.5606}

This one works much faster, but I also get a few error messages like this one

NIntegrate::maxp: The integral failed to converge after 50100 integrand evaluations. NIntegrate obtained -21.2913 and 1.3762138095540868` for the integral and error estimates.

Questions

  1. Is there a good way to compare these methods and make sure which results are reliable? I suspect this is due to a singularity in (part of) the integrated – since as T->0 the EllipticTheta function approaches to a sum of Dirac delta functions. Analytically, this does not seem to be a problem since the Exp(-y^2/(8T)) factor causes the integrand to become identical to zero. However, I imagine things are not as straight in numerics, but I also do not know how to overcome this hurdle.

  2. What can I do to speed up these computations? Especially, for generating and Manipulating the Plot3D of integral (or different variants of it) with {x,0,1},{xp,0,1}

  3. How to (numerically) take derivatives from integral w.r.t both x and xp? I both need to plot this derivative, as well as to integrate it against another kernel.

numerical integration – Multidimensional NIntegrate with Interpolating

I have a numerically evaluated function f(x,y) (it is impossible to write down analytical epxression for the function f) and array of points {x,y} with constant steps in x and y directions. I try to perform suggested strategy (see this question) My data is not pretty and Interpolation says me that InterpolationOrder should be set to All or to 1. If I use ->1, it returns

Interpolation::fememtlq: The quality -1.21606*10^-15 of the underlying mesh is too low. 
The quality needs to be larger than 0.`.

Then, I obtain interpolation function and try to perform NIntegrate. However, when I try to do this, it seems that Mathematica kernel crashes.

To be honest, I know nothing about numerical integration but I think that Method specification in NIntegrate can help. I can provide my data and result of “naive” calculation (just replace integrate by sum over points).

I try to understand which method of NIntegrate should be used. I plot DensityPlot of interpolating function and (may be) it helps.

enter image description here

why can’t i integration and differentiation?

Please kindly help solve this question for me. I need it quite urgent

continuous integration – How to test dependencies between microservices?

We have two microservices A and B that are developed inhouse.
A provides an API that B consumes.

Let’s assume that B has some specific integration tests that ensure that B is compatible with A.

We want these test to be run whenever the developer team from A commits a change.

Currently each team has a separate Jenkins and with Jenkins-terminolgy we would like to trigger a particular stage in B-pipeline whenever A changes.

We don’t think our scenario is possible with each team having a separate Jenkins. So we would like to evaluate other CI-Tools that maybe support our scenario.

Any ideas?

Microservice integration with an external legacy system

We are decomposing a monolithic legacy system into microservices. As we do so, we can’t completely remove reliance on some of the data in the legacy system that’s required for each microservice.

Should each microservice integrate directly with the legacy database to read and write data under its domain or should we have a single integration service responsible for fetching and writing data to the legacy system?

numerical integration – Using NIntegrate to reproduce NProbability over joint Gaussian distribution

Consider a random vector {s,c} with a bivariate normal distribution. For a vector of positive scalars {a, ß, σz}, I’m interested in calculating (numerically) the probability

NProbability(c < (1 - CDF(NormalDistribution( ß*(s - c), σz), a),{s,c} (Distributed) BinormalDistribution({μs, μc}, {σs, σc}, ρ))

Is there a way to write this same calculation using only NIntegrate?

NB. On my first attempt, I tried re-writing the probability, solving for s on one side of the inequality

NIntegrate(PDF(BinormalDistribution({μs, μc}, {σs, σc}, ρ),{s,c}),{s, c + (a-σz*InverseCDF(NormalDistribution(),1-c))(ß)^-1,(Infinity)},{c,-(Infinity),(Infinity)})

…but this approach doesn’t work because the computation gets stuck with InverseCDF(NormalDistribution(),1-c) for c below zero or above one.

web hosting – Will changing IP address cause any integration or security problems?

We have an ecommerce website that uses PayPal and SagePay to process payments. Our host moved our site to a new server recently, resulting a change of IP address. This caused SagePay to temporarily stop working until we registered the new IP address with them.

Will the change of IP address cause any other technical or security issues?

I’m not worried about SEO, I’m more concerned about technical/integration problems. It’s a WordPress site and we use Woocommerce, Analytics, Adwords, LiveChat Inc, SagePay, PayPal, Zapier, Salesforce/Formstack, Facebook/LinkedIn tracking codes etc.

numerical integration – Definite integral Closed-form or approximationsimplication or a plot diagram shows the variation based on $a$ and $b$

I am not professional in Mathematica and I am struggling to understand how the following function acts

$f(x)=int_{z=0}^infty int_{y=0}^{z} frac{x}{ sqrt{1-left(frac{z^2+y^2-x^2}{2 z y}right)^2}}
e^{- a y^2} , dy,
e^{-b z^2} , dz$

where $a>0$ , $b>0$, and obviously $|z-y| leqslant x leqslant y+z$.

I tried the following code

Integrate(x*e^(-a*y^2)*e^(-b*z^2))/(sqrt(1-((z^2+y^2-x^2)/(2*z*y))^2)) , {z, 0, Infinity}, {y, 0, z}, Assumptions -> a > 1 && b>1 && abs(y-z)(LessSlantEqual)x(LessSlantEqual)y+z)

However, I did not get any understandable result!
I would appreciate it if anyone can help me to find a closed-form or approximation of this integral.
I want to understand how this function varies with respect to $a$ and $b$?
Any kind of plot diagram can also help.

Thanks