nginx – Production server much slower then local test server when sending large request

I am running a django app and I am trying to send an API POST-request which is pretty big to my production server.

Now the problem is that what takes 4seconds locally takes 1minute in production. That might not sound like much but I am planning on sending this request 100 of times and every second counts.

I circled in on the problem and I think it might be an nginx configuration problem but I can’t be certain. Here is my troubleshooting process with respective code:

I have a big dictionary dataset_dict = {1:1234, 2:1244 ... 525600: 124345662}, so that means roughly half a million entries.

I send this and measure the time of my post-request:

            dataset_dict = {1:1234, 2:1244 ... 525600: 124345662}

            data = {
                "element": name_element,
                "description": f"description of element",
                "type_data": "datatype",
                "data_json": dataset_dict,
            }

            start = datetime.datetime.now()
            requests.request("post", url="myendpoint", json=data)
            end = datetime.datetime.now()
            runtime = end - start
            print("time-post-request:", runtime)

This takes 4 seconds locally and 50 seconds in production.

So I keep going and I measure the time of only the server code. With that I mean only the code that is executed in my view. I use raw-SQL to achieve maximum performance

    start_time = datetime.datetime.now()
    cursor = connection.cursor()
    data_json = json.dumps(request.data("data_json"))
    
    ##......code shortened for clarity

    cursor.execute(
        "INSERT INTO sql_table(data_json) VALUES ('{}')".format(data_json)
    )
    end_time = datetime.datetime.now()
    runtime = end_time - start_time
    
    print("success, time needed", runtime)

    msg = {"detail": "Created successfully"}
    return Response(msg, status=status.HTTP_201_CREATED)

This code on the server needs 3seconds locally and only 2seconds in production.

So my question is now: Where do the 56 seconds go?

I am inferring that I can exclude postgreSQL settings since the data injection seems to work quite fast.

Nginx settings could be a good start to look, so I was monitoring the server log and I got warn) 28#28: *9388 a client request body is buffered to a temporary file /var/cache/nginx/client_temp/0000000060,

In the official doc I read:

If the request body size is more than the buffer size, then the entire (or partial) request body is written into a temporary file

So I assume I lose time because nginx writes the request to disk. I adjust the request body size to 4GB. I run it again and it takes almost the same amount of time (55 seconds), but no warning anymore….

Is there any other screws I could adjust to get the performance up and the creation time down to what I have locally?? Shouldn’t production servers in general be faster than local dev servers?? Maybe it’s the internet connection?

So my main question: How could I increase the performance so I could get to a comparable time (seconds) for my data creation?

Specs:

Linux server 
RAM: 16GB 
CPUS: 4

I will post my nginx settings. I am really no expert with nginx so any help how I could increase my performance for this use-case is highly appreciated also if it doesn’t solve the problem.

Nginx:

worker_processes auto;

events {
    worker_connections 1024;
    use epoll;
    multi_accept on;
}


http {

    ...
    sendfile        on;
    tcp_nopush      on;
    tcp_nodelay    on;
    send_timeout       90;
    keepalive_timeout  90;
    fastcgi_read_timeout 120;
    proxy_read_timeout 120;
    fastcgi_buffers 8 128k;
    fastcgi_buffer_size 128k;
    client_body_timeout 120;
    client_body_buffer_size 4G;
    client_header_buffer_size 1k;
    large_client_header_buffers 4 8k;
    client_header_timeout 120;
    client_max_body_size 5G;

    reset_timedout_connection on;
    types_hash_max_size 2048;
    server_tokens off;

    gzip  on;
    gzip_static on;
    gzip_min_length 512;

}

Any more info needed, I’ll be happy to post it.

magento2 – Magento 2 Ajax form key invalid request

I’m trying to send a form from a modal and also an image from that form.

the form_data works fine with only serialize(), but I need to attach the image to data, so I did this:

   click: function (data) {
                    var form_data = new FormData();
                    var form = $("#bss-refund-form").serialize();

                    form_data.append('form', form);
                   
                
                    $.each($('#evidence')(0).files, function(i, file) {
                        form_data.append('evidence', file);
                       });

                    

                    if($('#bss-refund-form').valid()) {

                        $.ajax({
                            showLoader: true,
                            url: url,
                            type: 'POST',
                            data: form_data,
                            processData: false,
                            contentType: false
                        })
                            .done(function () {
                                $("#bss-refund-modal").modal('closeModal');
                                $("#bss-refund-form")(0).reset();
                                location.reload(true);
                                console.log("ok" + form_data);

                            })
                            .fail(function () {
                                $("#bss-refund-modal").modal('closeModal');
                                console.log("bad" + form_data);
                            });
                    }
                }

in the Controller here is the form_key validation.

if (!$this->formKeyValidator->validate($this->getRequest())) {
            $this->messageManager->addErrorMessage("Invalid request!");
            return $resultRedirect->setPath('customer/account/');
        }

reference request – $mathcal{D}_lambda$ modules $mathcal{D}_{0}$ modules equivalence

Fix $G$ a finite dimensional reductive group and $lambda$ a weight. Apparently the category of $mathcal{D}_lambda$ modules on $G/B$ is equivalent to $mathcal{D}_0$ modules on $G’/B’$ for a different group $G’$, whose Weyl algebra $W’$ are the elements $win W$ with $wlambda-lambda$ integral. Apparently this is due to Lustzig, where it is covered in the language of monodromic sheaves.

Question: is there a modern reference for this?

reference request – A Dynkin type classifcation result in linear algebra

Let $G$ be a finite directed acyclic graph. The Cartan matrix $C_G=C$ of $G$ is defined as the matrix with rows and colums indexed by the vertices of $G$ and $c_{i,j}$ counts the number of paths from $i$ to $j$.
The Coxeter matrix of $G$ is defined by $Phi_G=Phi=- C^{-1}C^T$.
Call a matrix $M$ periodic if $M^k=mathrm{id}$ for some $k geq 1$. Call $G$ periodic if $Phi_G$ is periodic.
The following gives a Dynkin type classification in linear algebra/graph theory:

Theorem: $G$ is periodic if and only if $G$ is of (simply laced) Dynkin type.

Question 1: Is there a direct (and easy) linear algebra proof of this result (in the literature)?

Question 2: The Coxeter matrix is well known from representation theory, but does it also appear in a purely graph theoretic context in a natural way?

code request – Interpolation not working

This is probably a very silly question but I am trying to use Interpolation for the following data:

  data={{0.965251, 0.}, {3.0888, 0.}, {5.98456, 0.}, {9.26641, 0.}, {12.5483,
   0.}, {15.8301, 0.}, {18.1467, 0.}, {21.8147, 0.}, {26.2548, 
  0.}, {30.695, 1.12676}, {33.3977, 1.69014}, {36.8726, 
  2.53521}, {39.3822, 3.38028}, {42.8571, 3.94366}, {45.7529, 
  5.35211}, {48.8417, 6.76056}, {51.5444, 8.73239}, {54.0541, 
  10.9859}, {55.7915, 13.2394}, {58.1081, 16.9014}, {59.0734, 
  19.1549}, {61.0039, 22.8169}, {62.1622, 26.4789}, {63.3205, 
  30.4225}, {64.2857, 34.3662}, {65.0579, 38.3099}, {65.8301, 
  42.8169}, {66.7954, 47.8873}, {67.3745, 53.8028}, {68.3398, 
  60.5634}, {68.9189, 65.6338}, {69.112, 71.5493}, {69.6911, 
  78.0282}, {70.2703, 84.507}, {70.6564, 88.4507}, {70.6564, 
  92.3944}, {71.4286, 97.1831}, {72.5869, 100.563}, {73.7452, 
  98.3099}, {74.5174, 92.6761}, {74.7104, 87.3239}, {75.0965, 
  82.2535}, {75.2896, 77.4648}, {75.6757, 72.1127}, {76.0618, 
  63.3803}, {76.6409, 54.6479}, {77.027, 47.3239}, {77.4131, 
  40.}, {78.3784, 35.2113}, {78.5714, 29.8592}, {80.1158, 
  24.2254}, {81.4672, 22.2535}, {82.8185, 19.7183}, {84.3629, 
  18.8732}, {87.2587, 18.5915}, {91.6988, 18.8732}, {94.5946, 
  18.5915}, {98.2626, 18.3099}, {100., 18.3099}, {120., 18.3099}};

Interpolation[data]

but for some reason it does not Interpolated correctly. If I plot the data it looks fine. Can someone tell me why Interpolation doesn’t correctly fit this data?

I get the following errro message:

InterpolatingFunction::dmval: Input value {0.00245143} lies outside the range of data in the interpolating function. Extrapolation will be used.

How can i use "Send an HTTP request to SharePoint" to only break inheritance for ONE folder and then add edit permission for just THAT folder.?

i have a subfolder within a folder and I have the path for a specific folder. Need to break inheritance for THAT folder and then add a person’s email address with owner permissions for THAT folder. Tried this, but it does it for ALL folders

reference request – Books that teach other subjects, written for a mathematician

Say I am a mathematician doesn’t know any chemistry, but would like to learn it. What books should I read?

Or say I want to learn about Einstein’s theory of relativity, but I don’t even know much basic physics. What sources should I read?

I am looking for texts that teach subjects that are not mathematics, but I do not want to read through standard high school, undergraduate (and beyond) material. I am looking for recommendations of sources that teach a scientific theory from a basic level, but not from a basic mathematical level. Strong preference would be to concise, terse texts that are foundational but totally rigorous.

Not sure if these exist, but I often wish they did.

Magento 2: Pre-load key request fonts/Luma-Icons.woff2

How can I remove fonts/Luma-Icons.woff2 from google page speed insight. From which less file this font is being loaded.

enter image description here

After creating a page, the "The Request is blocked" message is displayed [closed]

Once I create a new page and save it, the following error is displayed.

screenshot

No errors are displayed between the Recent log messages.

How do I request Facebook overrule my post as false? It is my own personal video

I posted a year ago, a video of a major hail storm we went through that wrecked my daughters car. It was my own footage and had over 5 million views around the world…however for some reason Facebook has marked it as fake! Can anyone please advise how I dispute this and get it overturned? I’d really like to keep this in my Facebook memories. Many thanks. Lisa