ubuntu – Configure rsyslog on a haproxy1.7 container

I’m using HAProxy 1.7 which needs rsyslog to push logs to stdout. Once the container writes logs to stdout, programs like filebeat or fluentd will pickup.

Docker container has: Ubuntu 20.04, HAProxy 1.7.12 2019/10/25, rsyslog v8.2001.0

docker-entrypoint.sh

#!/bin/sh
set -e

readonly RSYSLOG_PID="/app/rsyslogd.pid"
start_rsyslogd() {
  rm -f $RSYSLOG_PID
  rsyslogd -n -i "/app/rsyslogd.pid"
}
# Start rsyslog
start_rsyslogd

# Launch HAProxy
./haproxy-start.sh "$@"

rsyslog part config in Dockerfile:

# Install & run setcap to enable non-root user to bind sockets
RUN apt-get update && apt-get install -y --no-install-recommends 
    rsyslog 
    rm -rf /var/lib/apt/lists/* && 
    setcap CAP_NET_BIND_SERVICE=+eip "$(which haproxy)" && 
    touch /var/log/haproxy.log  && 
    ln -sf /dev/stdout /var/log/haproxy.log

COPY docker-entrypoint.sh /app
COPY haproxy-start.sh /app

COPY haproxy-rsyslog.conf /etc/rsyslog.d/haproxy.conf
COPY rsyslog.conf /etc/rsyslog.conf

RUN chmod a+x /app/*.sh && 
    chown -R ${USER}:${GROUP} /app
USER ${USER}
WORKDIR /app
ENTRYPOINT ("/usr/local/bin/dumb-init","--","./docker-entrypoint.sh")
CMD ("-f", "./config/haproxy.cfg")

rsyslog.conf:

$ModLoad imudp
$UDPServerAddress 127.0.0.1
$UDPServerRun 5140
local1.* /var/log/haproxy.log
& ~

Error on haproxy container logs:

│ rsyslogd: No UDP socket could successfully be initialized, some functionality may be disabled.  (v8.2001.0)                                         │
│ rsyslogd: imudp: Could not create udp listener, ignoring port 514 bind-address 127.0.0.1. (v8.2001.0)                                               │
│ rsyslogd: imudp: no listeners could be started, input not activated.  (v8.2001.0)                                                                   │
│ rsyslogd: activation of module imudp failed (v8.2001.0 try https://www.rsyslog.com/e/-3 )                                                           │
│ rsyslogd: could not remove supplemental group IDs: Operation not permitted (v8.2001.0 try https://www.rsyslog.com/e/2432 )                          │
│ rsyslogd: run failed with error -2432 (see rsyslog.h or try https://www.rsyslog.com/e/2432 to learn what that number means)                         │
│ stream closed    

It looks like a permissions issue but not sure how to configure rsyslog completely in userspace. Any suggestions around it would be great! TYA!

Ref: https://ops.tips/gists/haproxy-docker-container-logs/

Cannot remove or reinstall Java SDK on Ubuntu 18.04

I’m running Ubuntu 18.04 and cannot remove or reinstall the Java 11 SDK. I originally installed the OpenJDK version and things seemed to be fine but I wanted to install rsyslog today and couldn’t because of some Java 11SDK error.

I have tried running sudo apt remove openjdk-11-jdk and sudo apt purge openjdk-11-jdk

Here is the output from the purge command:

$ sudo apt purge openjdk-11-jdk
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package 'openjdk-11-jdk' is not installed, so not removed
The following packages were automatically installed and are no longer required:
  libice-dev libpthread-stubs0-dev libsm-dev libx11-dev libx11-doc libxau-dev libxcb1-dev libxdmcp-dev libxt-dev
  linux-image-4.15.0-122-generic linux-modules-4.15.0-122-generic x11proto-core-dev x11proto-dev xorg-sgml-doctools
  xtrans-dev
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 28 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Setting up oracle-java11-installer-local (11.0.9-1~linuxuprising0) ...
Before installing this package,
please download the Oracle JDK 11 .tar.gz file
with the same version as this package (version 11.0.4),
and place it in /var/cache/oracle-jdk11-installer-local,

E.g.:
sudo mkdir -p /var/cache/oracle-jdk11-installer-local
sudo cp jdk-11.0.4_linux-x64_bin.tar.gz /var/cache/oracle-jdk11-installer-local/
sha256sum mismatch jdk-11.0.9_linux-x64_bin.tar.gz
Oracle JDK 11 is NOT installed.
dpkg: error processing package oracle-java11-installer-local (--configure):
 installed oracle-java11-installer-local package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 oracle-java11-installer-local
E: Sub-process /usr/bin/dpkg returned an error code (1)

It says it’s not installed but you can see it is with this command:

$ java –version
openjdk 11.0.9.1 2020-11-04
OpenJDK Runtime Environment (build 11.0.9.1+1-Ubuntu-0ubuntu1.18.04)
OpenJDK 64-Bit Server VM (build 11.0.9.1+1-Ubuntu-0ubuntu1.18.04, mixed mode, sharing)

I also download the jdk-11.0.4_linux-x64_bin.tar.gz and put it into the /var/cache/oracle-jdk11-installer-local as suggested in the output but it makes no difference.

Appreciate any thoughts or suggestions. I just want to get JDK 11 working properly at this point!

ubuntu – Nginx rate limiting on unique IPs

we’ve been dealing with constant attacks on our authentication url, we’re talking millions of requests per day, my guess is they are trying to brute force passwords.

Whenever we would block the IP with the server firewall, few seconds later the attacks would start again from a different IP.

we ended up implementing a combination of throttling through rack-attack plus custom code to dynamically block the IPs in the firewall. But as we improved our software’s security, so did the attackers, and now we are seeing every request they make is done from a different IP, one call per IP, still several per seconds, not as many but still an issue.

Now i’m trying to figure out what else can i do to prevent this, we tried recaptcha but quickly ran out of the monthly quota and then nobody can login.

I’m looking into Nginx rate limiter but from what I can see it also uses the IP, considering they now rotate IPs for each request, is there a way that this would work?

Any other suggestions on how to handle this, maybe one of you went through the same thing?

Stack: Nginx and Rails 4, Ubuntu 16.

linux – Are there any free Ubuntu Servers that I can run my code on remotely?

Most cloud providers offer some sort of free tier, so there is usually an opportunity to run a Linux for no (or very little) outlay.

For example, AWS, Azure and Google

Of course the high specification or more powerful a server is, you will likely nudge beyond the free tiers. But the cost/benefit of this may be worthwhile if you want your model to finish sooner.

The top tip for Cloud infrastructure is to make sure you turn off or delete what you are not using.
Hope that help you get started.

Maximum password lenth for user accounts on Ubuntu

I have a VPS hosted at a remote location. Obviously the host can login directly (without SSH or any keys) on this box and I want to make brute force password guessing as hard as possible for a bad admin.

What is the maximum password length for user accounts in Ubuntu?

(I’m logging in with a 4K RSA SSH key so I don’t care about “user friendliness” for manual logins.)

networking – how to save static routes permanently in ubuntu

The current configuration:

server1:    
sudo route add -host 10.0.1.2 dev enp131s0f0
sudo route add -host 10.0.1.3 dev enp131s0f1

server2:    
sudo route add -host 10.0.1.1 dev enp131s0f0
sudo route add -host 10.0.1.3 dev enp131s0f1

server3:    
sudo route add -host 10.0.1.1 dev enp131s0f0
sudo route add -host 10.0.1.2 dev enp131s0f1

This configuration on both sides will be lost if any server is rebooted or the cable is unplugged and replugged.

Saving them in /etc/rc.local does not work for the above situations.

So, how to save them permanently for both netplan and NetworkManager. I have both Ubuntu desktop and Ubuntu server installed.

ubuntu – Can connect to Postgres remotely, but not from localhost

I’m having some trouble connecting to my server’s Postgres database from an API that is running on the same machine as Postgres.

I’ve been following this Google Cloud Guide to set it up.

  1. I’ve edited pg_hba at sudo nano /etc/postgresql/12/main/pg_hba.conf.
    I’ve added my own IP at the bottom of it so I can connect to it from my machine, and ensured that the file allowed for localhost connections as well:
# "local" is for Unix domain socket connections only
local   all             all                                     peer
# IPv4 local connections:
host    all             all             127.0.0.1/32            md5
# IPv6 local connections:
host    all             all             ::1/128                 md5
  1. I edited /etc/postgresql/12/main/postgresql.conf and set listen_addresses = '*'.

  2. I’ve restarted the server.

I can now successfully connect remotely (from my machine) by using the string:

postgres://(username):(password)@(ip of server):5432/(database name)

However, if I try to use the following with an API that is running on the same server:

postgres://(username):(password)@<localhost or 127.0.01>:5432/(database name)

No connection can be established. I don’t actually get an error, which leads me to believe I either

  1. Have the string written wrong
  2. I’m missing something to allow that connection

Any help would be really appreciated. I’m running this on a Gcloud Compute Engine Ubuntu instance, although I don’t think that makes much difference. I have set all the rules correctly in my firewall settings as well.

hardware – Acer Aspire 5 and Ubuntu 20.04 hangs on boot

My Aspire 5 515-54G-73N7 is running Ubuntu 20.04. Until some days ago, I had no problems during booting.

Now, almost every time, but not always, it hangs during boot. Then it either shows a blank black screen with a blinking cursor or only the following lines:

Bluetooth: hci0: command 0xfc09 tx timeout
Bluetooth: hci0: Failed to send firmware data (-110)
Bluetooth: hci0: sending frame failed (-19)

The kernel version is 5.8.0-36 (I tried to boot an older kernel but the problem also occurs there). However, I can boot into recovery mode.

I ran a MemTest, it showed no errors.

The following “red lines” are logged in kernel.log:

(    0.334085) pci 0000:02:00.0: BAR 6: failed to assign (mem size 0x00080000 pref)
(    0.424297) Initramfs unpacking failed: Decoding failed
(    0.482633) pcieport 0000:00:1c.4: DPC: error containment capabilities: Int Msg #0, RPExt+ PoisonedTLP+ SwTrigger+ RP PIO Log 4, DL_ActiveErr+
(    0.483215) pcieport 0000:00:1d.4: DPC: error containment capabilities: Int Msg #0, RPExt+ PoisonedTLP+ SwTrigger+ RP PIO Log 4, DL_ActiveErr+
(    0.540227) RAS: Correctable Errors collector initialized.
(    2.463845) EXT4-fs (nvme0n1p2): re-mounted. Opts: errors=remount-ro
(    2.724810) iwlwifi 0000:00:14.3: Direct firmware load for iwlwifi-QuZ-a0-hr-b0-56.ucode failed with error -2
(    2.727538) iwlwifi 0000:00:14.3: Direct firmware load for iwl-debug-yoyo.bin failed with error -2
(    3.041233) thermal thermal_zone1: failed to read out thermal zone (-61)

Is there any hint to what’s causing the problem?

And I assume there is a (new?) heat/fan problem. The fan is running (almost) constantly now (not only when Ubuntu is up but also in EFI), although there is almost no load.

Oracle VirtualBox for Ubuntu detecting avx & avx2 for one CPU but not the other

I have been looking into this issue for days now. In short, Oracle VirtualBox started supporting avx & avx2 since version 5.0.3. Right now, it’s on 6.1.16, so way ahead.

I had many problems installing TensorFlow in my VirtualBox Ubuntu on this desktop with Intel Core i7 4790K. I boiled it down after a lot of investigation into avx/avx2 not being enabled in the guest. My CPU flags prove this when I type more /proc/cpuinfo | grep flags into the bash shell.

I have already looked at possible solutions (1, 2, 3) and run them. They show Key: VBoxInternal/CPUM/IsaExts/AVX2 1 as expected. But it still doesn’t show up when I type the more /proc/cpuinfo | grep flags into the bash shell.

The most important difference from the previous questions is it DOES work and avx(2) DO show up in my newer laptop with i7-8565U and everything works perfectly. I have made sure Intel Virtualization is enabled on my BIOS, switched to KVM Paravirtualization, etc. done everything to make them identical. However, it’s not working in my 4790K computer.

I have confirmed BOTH CPUs support AVX according to their specifications.

wordpress mu – OpenSSL Configuration for Ubuntu 20.04 LAMP& WP Multisites

Question: How do I apply 1 OpenSSL certificate to all websites located in html directory?
Problem: When visiting https://localhost.site1.com or https://localhost.site2.com only index.html located at /var/www/html/index.html is displayed because default-ssl.conf document root is /var/www/html/

I have 2 wordpress multisites (and other sites) located in /var/www/html/:

/var/www/html/site1.com

and

/var/www/html/site2.com

In my default-ssl.conf I have:

<IfModule mod_ssl.c>
<VirtualHost _default_:443>
    ServerAdmin info@dummy.com
    ServerName localhost
    ServerAlias localhost

    DocumentRoot /var/www/html/
    
    ErrorLog ${APACHE_LOG_DIR}/localhost.error.log
    CustomLog ${APACHE_LOG_DIR}/localhost.access.log combined

            SSLEngine on
    SSLCertificateFile  /etc/ssl/certs/ssl-cert-snakeoil.pem
    SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
    
    #SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire
    <FilesMatch ".(cgi|shtml|phtml|php)$">
            SSLOptions +StdEnvVars
    </FilesMatch>
    <Directory /usr/lib/cgi-bin>
            SSLOptions +StdEnvVars
            DirectoryIndex index.php
            AllowOverride All
            Order allow,deny
            Allow from all
            Require all granted
    </Directory>

    #   Similarly, one has to force some clients to use HTTP/1.0 to workaround
    #   their broken HTTP/1.1 implementation. Use variables "downgrade-1.0" and
    #   "force-response-1.0" for this.
      BrowserMatch "MSIE (2-6)" 
            nokeepalive ssl-unclean-shutdown 
            downgrade-1.0 force-response-1.0

</VirtualHost>
</IfModule>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

In my /etc/hosts file I have:

127.0.1.1   excalibur
127.0.0.1   localhost 
127.0.0.1   localhost.site1.com *.localhost.site1.com   # mainsite url
127.0.0.1   subsite-a.localhost.site1.com   
127.0.0.1   subsite-b.localhost.site1.com
127.0.0.1   subsite-c.localhost.site1.com

127.0.0.1   localhost.site2.com *.localhost.site2.com   # mainsite url

The vhost for site1.com contains:

    <VirtualHost *:80>

    ServerName localhost.site1.com 
    ServerAlias www.localhost.site1.com
    
    # If this is the default configuration file we can use: 'ServerName localhost' or also 'ServerAlias localhost'.

    ServerAdmin info@dummy.com

    ErrorLog ${APACHE_LOG_DIR}/localhost.site1.com.error.log
    CustomLog ${APACHE_LOG_DIR}/localhost.site1.com.access.log combined

    DocumentRoot /var/www/html/site1.com
    
    <Directory /var/www/html/site1.com>
        Options None FollowSymLinks
        # Enable .htaccess Overrides:
        AllowOverride All
        DirectoryIndex index.php
        Order allow,deny
        Allow from all
        Require all granted
    </Directory>

    <Directory /var/www/html/site1.com/wp-content>
        Options FollowSymLinks
        Order allow,deny
        Allow from all
    </Directory>
    
    
   SSLEngine on
   SSLCertificateFile /etc/ssl/certs/apache-selfsigned.crt
   SSLCertificateKeyFile /etc/ssl/private/apache-selfsigned.key

</VirtualHost>

And the vhost for site2.com contains:

    <VirtualHost *:80>

    ServerName localhost.site2.com
    ServerAlias www.localhost.site2.com
    
    # If this is the default configuration file we can use: 'ServerName localhost' or also 'ServerAlias localhost'.

    ServerAdmin info@dummy.com

    ErrorLog ${APACHE_LOG_DIR}/localhost.site2.com.error.log
    CustomLog ${APACHE_LOG_DIR}/localhost.site2.com.access.log combined

    DocumentRoot /var/www/html/site2.com
    
    <Directory /var/www/html/site2.com>
        Options None FollowSymLinks
        # Enable .htaccess Overrides:
        AllowOverride All
        DirectoryIndex index.php
        Order allow,deny
        Allow from all
        Require all granted
    </Directory>

    <Directory /var/www/html/site2.com/wp-content>
        Options FollowSymLinks
        Order allow,deny
        Allow from all
    </Directory>
    
   SSLEngine on
   SSLCertificateFile /etc/ssl/certs/apache-selfsigned.crt
   SSLCertificateKeyFile /etc/ssl/private/apache-selfsigned.key
   
</VirtualHost>

Any tips?