NETSEC – Detecting unusual traffic in the Cloud using Flowlogs/Lambdas

This is a PoC I did awhile back and I lost the original content when porting over to my new domain. Essentially this is a PoC code that can be modified to detect the source, destination and port/protocol network communication between boundaries within your cloud VPC for “weird” or “unusual” traffic.

https://github.com/secSandman/lambda_netflows/blob/master/lambda-netflow-data-loss.js

For example, you may not normally allow communication from some development subnet/vpc —> production subnet/vpc or you may not allow egress/ingress to/from the internet to some subnets/vpc. The exact scenario may vary but in cases where’d you’d not normally expact to see data movement or remote administration you can passively parse cloud log events and notify your ops and security staff for further investigation.

I lost a lot of my performance testing data but if I remember correctly, I believe I got the node.js code execution between 30-60 milliseconds but I could be wrong. The code did once execute in the hundreds but I remember removing some unnecessary iterations that were causing additional processing. There may be other bugs but the functionality did work to demonstrate the concept.

These are the variable that help define your particular use-case, meaning “unusual traffic” . you may change based on your PoC

var privips = ['204.2.134','0','10','100.64','127','169.254','172.16','192','192.0.0.9','192.0.0.170','192.0.0.171','192.0.2','192.31.196','192.52.193','192.88.99','192.168','192.175.48','198.18','198.51.100','203.0.113'];

var interNets = ['172.31.1', '172.31.2','172.31.44'];
var badports = [20,21,22];

I may have found that there were many duplications flow logs that were all being processed creating duplications of lambda executions and unnecessary SNS notifications. I think I found way to de-dupe these events to minimize the processing and duplicate notifications. This is just a guess.

function diffArray(object) {
  var R1 = [];
  //R1 will contain anything unique in arr1
  R1 = object.array1.filter(function(elem) {
    return object.array2.indexOf(elem) > -1;
  });
  return R1;
}

If I remember correctly, I believe I used some type of set theory between the use-cases array values and the de-duped vpc flow log cloud object values. For some reason, I feel like I did this in a way to try and reduce the processing logic needed to find a match/negative match.

unZip(payload);
      function unZip(payload) {
        zlib.gunzip(payload, (err, res) => {
        if (err) {
            return callback(err);
        }
        const parsed = JSON.parse(res.toString('utf8'));
        logEvents(parsed);
        
        let setAB = {
            array1: srcIpFromFlow,
            array2: srcPortFromFlow
        }
        
        let setBC = {
            array1: diffArray(setAB),
            array2: destIpFromFlow
        }
        
        let setD = diffArray(setBC);

        setD.forEach(function(element) {
            let snsObj = parsed.logEvents[element];
            snsnotice(snsObj)
        });
    })};
    

Send the event that matches a rule as a email or text for further investigation.

        function snsnotice(snsObj){
         const data = {
               "default": JSON.stringify(snsObj) 
            };
 
          var smsparams = {
          Message: JSON.stringify(data),
          MessageStructure: 'json',
          Subject: 'Potential data loss',
          TargetArn: '<Your SNS TPOIC ARN'
          };
          
          sns.publish(smsparams, function(err, data) {
          if (err) console.log(err, err.stack); // an error occurred
          else     console.log(data);           // successful response
          });
    }          
};


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s