Securing APIs used by remote devices

We have desktop software that we install and run on remote, unattended machines. The software needs to hit API endpoints on our server. What is the best way to secure those endpoints to ensure that only our software can call them? We may or may not have admin rights to that machine. Others (staff or 3rd party individuals) may have access to that machine at an admin level.

Can’t SCP from Windows remote to MacOS

I have ran into an issue where I cannot use scp from my Mac laptop to copy files from my Windows PC.

The command I am running on MacOS:
scp user@pcname:UsersWilltreemnger Documents/treemnger*

It returns the same output as scp --help.

*‘user’ and ‘pcname’ are placeholders

security – How we can secure the ClientID & ClientSecret inside our remote event receivers (Inside SharePoint Provider hosted app)

I have developed many remote event receivers and host them inside azure web apps. now the web.config file inside those remote event receivers contain the ClientId & ClientSecret, as follow:-

 <appSettings file="custom.config">
    <add key="ClientId" value="e***7" />
    <add key="ClientSecret" value="h***g=" />
  </appSettings>

Now if a hacker or an end user found those values inside the project code, then the user can control all the sites, as when we register the remote event receivers we grant them full control on the site collection.. so any advice how we can secure those details? so if someone access the source code of the RER then she/he can not view those details?

Thanks

Incremental Remote Backups Using rsnapshot

There’s an old adage about backups:

There are two kinds of people: people who’ve never lost data, and people who’ll never lose data again.

If you’ve ever experienced data loss, you will instantly become passionate about backups.  To prevent bad experiences with your data, you want backups that are comprehensive, manageable, versioned, automated, and secure.  Let’s break that down:

  • comprehensive: They should include everything by default.  It’s certainly legitimate to exclude OS directories, temp files, etc. but you don’t want a system where you have to manually add directories as you add applications and data.  Inevitably, you’ll forget something and not know it until it’s too late.
  • manageable: If you have a 1TB server and take a full backup every day and retain them for a month, that’s 30TB in a month.  You need a system that allows for regular pruning.
  • versioned: If you have a system that simply copies everything from system A to system B once a night, that’s better than nothing, but on Monday you trash a file and don’t notice it until Thursday, you can’t recover.
  • automated: Because humans are lazy.
  • secure: It’s annoying to be hacked.  It’s heartbreaking to find the hacker also destroyed your backups.

In this tutorial, we’ll show you how to setup backups using rsnapshot.  Quoting rsnapshot.org:

rsnapshot is a filesystem snapshot utility based on rsync. rsnapshot makes it easy to make periodic snapshots of local machines, and remote machines over ssh. The code makes extensive use of hard links whenever possible, to greatly reduce the disk space required.

This means if you have 500MB of files, you want to retain 30 days’ backups, and your change rate is 10% over that period, you don’t need 30 * 500 = 15,000MB but rather only 550MB.  Beautifully, you still have point-in-time recovery (depending on your backup schedule) throughout that period.

In this tutorial, we’ll setup the following:

  • server1.lowend.party has a directory called /data with lots of valuable files.
  • We want to back it up to backup.lowend.party using a scheme of hourly/daily/weekly/monthly backups.  These are stored in /backups/server1.lowend.party
  • backup.lowend.party has other hosts it backs up as well.
  • We’re using passwordless ssh keys for authentication so we can run everything out of cron.

Before we start, there’s one more key concept.

I’ve long advocated pull backups.  In other words, the backup server comes along and backs up the client.  In this scenario, backup.lowend.party initiates the backups and contacts server1.lowend.party to get the data.  This is in contrast to push backups, where server1.lowend.party contacts backup.lowend.party and pushes the backups to it.

What’s the difference?  Imagine server1 is hacked.  If we’re using push backups, it would be trivial for the hacker to use the passwordless ssh keys to nuke the backups as well.  In a pull-based model, backup.lowend.party can authenticate to server1, but not vice-versa, so the hacker is out of luck.

On Debian, it’s as easy as

apt install rsnapshot

rsnapshot’s config lives in /etc/rsnapshot.conf.  I recommend making a backup of it before you start changing things:

mv /etc/rsnapshot.conf /etc/rsnapsnap.conf.default

There are different philosophies about how to setup rsnapshot configs.  I prefer to have a separate config file for each client (system being backed up).  If you only have one system to backup, this is not necessary.  You can backup multiple systems in one config file, but you lose some flexibility.  Experiment and decide which you like.  In my case, I do this:

cp /etc/rsnapshot.conf.default /etc/rsnapshot.conf.server1

Now modify as follows.  Important Note: rsnapshot.conf requires TABs between elements.  So “cmd /usr/bin/ssh” is “cmd<TAB>/usr/bin/ssh”.

Enable remote backups:

cmd_ssh /usr/bin/ssh

Add these backup intervals:

interval hourly 6
interval daily 7
interval weekly 4
interval monthly 3

I’m using a passwordless ssh key stored in /root/.ssh/backup.  I also use a different ssh port.  So make this change:

ssh_args -p 8989 -i ~/.ssh/backup
These two commands are for reporting (see below):

rsync_long_args --stats --delete --numeric-ids --relative --delete-excluded
verbose 4

Now I tell rsnapshot where to save backups:

snapshot_root /backups/server1.lowend.party/

Finally, I add the backup definition:

backup root@server1.lowend.party:/data/ .

This will keep files in /backups/server1.lowend.party/hourly.0, etc.

I want to exclude /data/cache on my backups:

exclude_file /etc/rsnapshot.server1.exclude

And in that file I put:

- /data/cache/*

OK, we’re ready to go.  Now because I’m not using the default /etc/rsnapshot.conf name, I need to use the -c parameter for all rsnapshot commands.  Let’s start by testing the config:

root@backup:/etc# rsnapshot -c /etc/rsnapshot.conf.server1 configtest
Syntax OK

Now we can run a simulation:

root@backup:/etc# rsnapshot -c /etc/rsnapshot.conf.server1 -t hourly
echo 9633 > /var/run/rsnapshot.pid
mkdir -m 0755 -p /backups/hourly.0/
/usr/bin/rsync -a --stats --delete --numeric-ids --relative --delete-excluded 
--exclude-from=/etc/rsnapshot.server1.exclude --rsh=/usr/bin/ssh -p 8989 
-i ~/.ssh/backup root@server1.lowend.party:/data/ 
/backups/hourly.0/server1.lowend.party/
touch /backups/hourly.0/

One more thing to do.  I like to use rsnapshot’s reporting tool, so let’s enable it:

cp /usr/share/doc/rsnapshot/examples/utils/rsnapreport.pl /usr/local/bin
chmod 755 /usr/local/bin/rsnapreport.pl 

We’re good to go!

On server1, I have 547MB in /data, and 30MB in /data/cache which will be excluded:

root@server1:~# du -sm /data
547 /data
root@server1:~# du -sm /data/cache
30 /data/cache

Let’s run our first rsnapshot backup:

root@backup:/backups/server1.lowend.party# rsnapshot -c /etc/rsnapshot.conf.server1 hourly
Setting locale to POSIX "C"
echo 10012 > /var/run/rsnapshot.pid
mkdir -m 0755 -p /backups/server1.lowend.party/hourly.0/
/usr/bin/rsync -av --stats --delete --numeric-ids --relative 
--delete-excluded --exclude-from=/etc/rsnapshot.server1.exclude 
--rsh=/usr/bin/ssh -p 8989 -i ~/.ssh/backup 
root@server1.lowend.party:/data/ 
/backups/server1.lowend.party/hourly.0/.
receiving incremental file list
data/
<snipped>
data/cache/

Number of files: 10,982 (reg: 10,980, dir: 2)
Number of created files: 10,982 (reg: 10,980, dir: 2)
Number of deleted files: 0
Number of regular files transferred: 10,980
Total file size: 518,702,282 bytes
Total transferred file size: 518,702,282 bytes
Literal data: 518,702,282 bytes
Matched data: 0 bytes
File list size: 611,123
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 208,691
Total bytes received: 519,874,481

sent 208,691 bytes received 519,874,481 bytes 80,012,795.69 bytes/sec
total size is 518,702,282 speedup is 1.00
touch /backups/server1.lowend.party/hourly.0/
rm -f /var/run/rsnapshot.pid
/usr/bin/logger -p user.info -t rsnapshot(10012) /usr/bin/rsnapshot -c 
/etc/rsnapshot.conf.server1 hourly: completed successfully

Now I can also run that using the rsnapshotreport.pl script we setup.  If I do, the output will look like this (the TOTAL MB is a little different because I ran these at different times):

# rsnapshot -c /etc/rsnapshot.conf.server1 hourly | /usr/local/bin/rsnapshotreport.pl
SOURCE TOTAL FILES FILES TRANS TOTAL MB MB TRANS LIST GEN TIME FILE XFER TIME
--------------------------------------------------------------------------------------------------------------------
server1.lowend.party:/data/ 11982 1 564.81 46.10 0.001 seconds 0.000 seconds

Now if I continue running hourly backups, I see new directories being created in /backups/server1.lowend.party:

drwxr-xr-x 3 root root 4096 Jul 12 16:03 hourly.0
drwxr-xr-x 3 root root 4096 Jul 12 16:01 hourly.1
drwxr-xr-x 3 root root 4096 Jul 12 15:58 hourly.2

Interestingly, hourly.0 is 500-odd MB, will the rest are only 1MB.  Why?  Because hourly.1, hourly.2, etc. are simply hard links back to hourly.0.  This is a huge space savings.

If I nuke some files on server1’s /data and run another couple backups, you’ll see this:

root@backup:/backups/server1.lowend.party# du -sm *
526 hourly.0
39 hourly.1
1 hourly.2
1 hourly.3

rsnapshot is retaining data in hourly.1 because it’s needed to reconstruct the backups for that hour.

Setting up automated backups is as easy as putting jobs in cron.  For example:

MAILTO=you@somewhere.com
0 * * * * root /usr/bin/rsnapshot -c /etc/rsnapshot.conf.server1 hourly 2>&1 | /usr/local/bin/rsnapreport.pl
0 3 * * * root /usr/bin/rsnapshot -c /etc/rsnapshot.conf.server1 daily 2>&1 | /usr/local/bin/rsnapreport.pl 
0 3 * * 1 root /usr/bin/rsnapshot -c /etc/rsnapshot.conf.server1 weekly 2>&1 | /usr/local/bin/rsnapreport.pl
30 2 1 * * root /usr/bin/rsnapshot -c /etc/rsnapshot.conf.server1 monthly 2>&1 | /usr/local/bin/rsnapreport.pl

Now

raindog308

I’m Andrew, techno polymath and long-time LowEndTalk community Moderator. My technical interests include all things Unix, perl, python, shell scripting, and relational database systems. I enjoy writing technical articles here on LowEndBox to help people get more out of their VPSes.

disk encryption – Security against local attack for remote FDE decryption?

Is there any remote FDE decryption that is resistant to an attacker that has local physical access?

Tools like dracut-sshd need to store the private key used for the sshd server on the unencrypted boot partition, so a local attacker has the ability to become a MITM and sniff the decryption password.

Can using a TPM to protect the sshd key foil this attack?

Does clevis-tang have essentially the same problem? At the bottom of the tang README.md is this list of security considerations:

  1. Man-in-the-Middle
  2. Compromise the client to gain access to cJWK
  3. Compromise the server to gain access to sJWK's private key

Problem (1) is not a concern according to this document. I assume you avoid problem (3) by running the tang server on a FDE itself or storing the key on a HSM. Problem (2) sounds impossible to protect against if the attacker is local – is that correct?

The tang documentation stresses that the…

client protect cJWK from prying eyes. This may include device
permissions, filesystem permissions, security frameworks (such as
SELinux) or even the use of hardware encryption such as a TPM

Is the TPM option the only way to foil an attacker with physical access to the unencrypted boot partition?

Is there any work-around that allows remote unattended FDE decryption that a local attacker cannot compromise?

How to get the URL of remote video inserted by media field in block or paragraph twig?

How to get the URL of remote video inserted by media field in block or paragraph twig?
I am using the paragraphs module to create video galleries from youtube videos. When I use {{ content.video_gallery }} it’s working and showing the youtube video thumbnail but I want to show this video in a popup for which I have written a code in which I need only the URL of the youtube video.
I am searching for two days I found solutions for image and video but no solution found for remote video URL.

teamwork – How to get up to speed quickly in new SE role, while: fully remote (COVID-19), in a satellite office, with time zones/language barrier to primary devs

Echoing a common question here – how to get up to speed at a new company.

I am curious if any particular tips stand out for the not-planned-work-from-home from COVID transition, or with cases where your department speaks a different language (and is timezone separated) from other parts of the organization.

Current efforts are mostly around reviewing documentation, but given language barrier some documents are not immediately useful (poor quality automatic translation) or not easily searchable, as well as often finding I need to request access permissions from another person whos work is shifted 12 hours after mine (so I get access to X the next day, only to find related links Y and Z which I then need to request).

What is adclick remote address? [migrated]

netstat -a returns the information below.

What is the "adclick" address? There is any benefit from disconnect it? How to shutdown that connection?

postgresql – connect to remote PostgresDB through VPN

I am not very advanced DB Admin/User, know only basics. Though as developer I need to have a direct access to a production-deployed database by using my VPN configuration so that I wouldn’t need to disconnect from VPN on local machine. First of all this is not secure, secondly, the database setup is very simple, no sharding or any complex backuping etc. There is neither any complex authentication for the connection, just password-protected. The only problem I have is to have connection to the AWS-deployed db via DBeaver (or any other GUI-client). I am not aware of details like socks / http(s) etc. I know that is possible, but I do not know where to dig. Can please help me navigate the basics what I have to consider and how to accomplish the goal of connecting to the remote db without disconnecting from VPN ?

P.S. I searched the site for other similar questions, but they’re mostly unrelated or overcomplicated (no need for JVM or another client or another DB etc.) but can’t figure out the simple steps for myself.

development – I am no longer able to debug/run my SharePoint Remote Event Receiver locally using Ngrok

Last month the below steps were working well for me to debug and test a SharePoint online remote event receiver locally:-

  1. Open Ngrok.exe >> run the following command inside ngrok:-
 ngrok authtoken 3***e
 ngrok http --host-header=rewrite  57269
  1. register a new app:- @ https://.sharepoint.com/sites//_layouts/15/AppRegNew.aspx >> enter the above Ngrok urls inside the App Redirect URL & App Domain.

  2. Inside the _layouts/15/appinv.aspx >> I search for the above app using Client ID >> and enter the following:-

 <AppPermissionRequests AllowAppOnlyPolicy="true">
   <AppPermissionRequest Scope="http://sharepoint/content/tenant" Right="FullControl" />
 </AppPermissionRequests>
  1. Update the service’s web config with the above ClientID and ClientSecret

  2. Register the new remove event receiver as follow:-

 Add-PnPEventReceiver -List "Order Management" -Name "TasksRER" -Url http://cc6e945e82f6.ngrok.io/service1.svc -EventReceiverType ItemUpdated -Synchronization Asynchronous

But today when I tried the above steps it failed >> where inside my event receiver when I tried to get the SharePoint context >> I will get that the Context is null:-

  public void ProcessOneWayEvent(SPRemoteEventProperties properties)
         {
             var prop = properties;
             var listItemID = properties.ItemEventProperties.ListItemId;
             var listTitle = properties.ItemEventProperties.ListTitle;
             using (ClientContext context = Helpers.GetAppOnlyContext(properties.ItemEventProperties.WebUrl))
             {
                 context.Load(context.Web);
                 context.ExecuteQuery();

Here is a screen shot from Visual Studio with the error i am getting when trying to get the context:-

enter image description here

Any advice if anything has been changed which is preventing me from running the above steps? which were working well last month?
Thanks

here is the code for the GetAppOnlyContext

     public class Helpers
         {
             public static ClientContext GetAppOnlyContext(string siteUrl)
             {
                 try
                 {
                     Uri siteUri = new Uri(siteUrl);
                     string realm = TokenHelper.GetRealmFromTargetUrl(siteUri);
                     string accessToken = TokenHelper.GetAppOnlyAccessToken(TokenHelper.SharePointPrincipal, siteUri.Authority, realm).AccessToken;
        
                     return TokenHelper.GetClientContextWithAccessToken(siteUri.ToString(), accessToken);
                 }
        
                 catch (Exception ex)
                 {
                     Trace.TraceInformation("GetAppOnlyContext failed. {0}", ex.Message);
                 }
                 return null;
             }
        
             public static ClientContext GetAuthenticatedContext(string siteUrl)
             {
                 string userName = WebConfigurationManager.AppSettings.Get("AuthenticatedUserName");
                 string password = WebConfigurationManager.AppSettings.Get("AuthenticatedUserPassword");
                 return GetAuthenticatedContext(siteUrl, userName, password);
             }
        
             public static ClientContext GetAuthenticatedContext(string siteUrl, string userName, SecureString password)
             {
                 ClientContext ctx = new ClientContext(siteUrl);
                 ctx.Credentials = new SharePointOnlineCredentials(userName, password);
                 return ctx;
             }
        
             public static ClientContext GetAuthenticatedContext(string siteUrl, string userName, string password)
             {
                 SecureString securePassword = GetPassword(password);
                 return GetAuthenticatedContext(siteUrl, userName, securePassword);
             }
        
             private static SecureString GetPassword(string passwd)
             {
                 var secure = new SecureString();
                 foreach (char c in passwd)
                 {
                     secure.AppendChar(c);
                 }
                 return secure;
             }
        
             public static string EmptyIfNull(object obj)
             {
                 return obj == null ? "" : obj.ToString();
             }
         }
     }  

seem this statement string accessToken = TokenHelper.GetAppOnlyAccessToken(TokenHelper.SharePointPrincipal, siteUri.Authority, realm).AccessToken; inside my Helpers.GetAppOnlyContext will raise this error token request failed, as follow:-

enter image description here