New features of PAN-OS version 7

Quick summary of the awesome new features the version 7 of PAN-OS introduces:

1. TLS 1.2 is finally supported! So we can do proper SSL-decryption now. Hooray!
2. IKE v2 is finally supported! So the Azure Dynamic mode I struggled with in 2013 should now be possible. Hooray!
3. TACACS+ authentication is now supported! Wow! Happy days for big enterprise networks full of Cisco kit – one password less to remember!
4. ACC correlation engine – sounds like a really cool feature allowing to cross-reference Wildfire events with network activity (i.e. a host has received a file and then exhibited C&C traffic similar to that appearing during Wildfire sand-boxing). So no more manual looking through access logs after a malware slipped through. Should be a really good time saver for SOCs!
5. Global Find command allowing to search in one go through Rules, Objects, Profiles et. etc. etc. – really useful thing! Before that the only way of doing so was to go through the raw XML config. Hooray!
6. Deny action can now be specified in a rule (drop, reset client, reset server or reset both). Finally! You also can configure whether to send ICMP Unreachable message or not (before you ask – ICMP messages can be rate-limited of cause).
7. Tag browser – allows to properly filter rules by Tags and work with these filtered rules. Finally something useful for big rule sets! Before that pretty much the only thing you could do was to view rules containing a certain tag. Hooray!
8. QoS is now supported on Etherchannels (we getting there with etherchannels by looks of it – at some point PA introduced  Etherchannels on low-end platforms, then introduced LACP and now QoS – patience of early adopters of PAN technologies pays out :))
9. A bit of L3 load balancing – ECMP support – may be useful in some cases, I guess…
10. User-ID can now look into XFF header on HTTP. Finally! Happy days for those who configures their proxies to report requesters in XFF or those who does a load balancing with SNAT and injection of original IP into XFF

There are many other minor improvements but I appreciate only the ones above 🙂

Palo Alto – Bulk rule editing via API and scripting

Perhaps all serious admins of Palo Alto firewalls have heard about the REST API that PAN provides with their firewalls. Not all of them have tried to automate their work though :).

You may not need to work with API on a daily basis to perform routine firewall changes but if you happen to get involved with firewall migrations, bulk network changes and the like then the API is a must have!

I absolutely love it. You may actually be not very good at scripting but rest assured that the bicycle of PAN API scripting has already been invited for you. The bicycle is called PAN-Configurator and you can get it from GitHub. If the link ever changes the new one is likely to be referenced on PAN web site here.

GARTNER Magic Quadrant for Enterprise Network Firewalls 2015

I have just been pleased to find out that I have been working with the right firewall brands over the past 4 years (Palo Alto, Checkpoint and a little bit of Cisco):

How Palo Alto Wildfire and antivirus work with SMTP

Just a note on how to enable Wildfire on SMTP traffic and how it looks like on the wire.

Palo Alto AppID and F5 HTTP Health Monitors

It appeared that you cannot really enforce (using application override policy) standard application called “web-browsing” for a session that carries traffic non-compliant with HTTP specifications. I am not sure how strict the checks are but standard F5 http health monitor where the request is defined as

GET /\r\n

is detected as “unknown-tcp” regardless of any override policies. The only way to make Palo Alto firewall recognize this traffic as “web-browsing” is to add HTTP protocol specification as follows:

GET / HTTP/1.0\r\n

or

GET / HTTP/1.1\r\n

Thanks to awesome Palo Alto Support for the clue.

p.s. Uselessness of the default F5 HTTP health monitor should be a subject of a separate post 🙂

Palo Alto – stale sessions blocking VPN and NetFlow traffic

I have just had to troubleshoot an interesting issue with Palo Alto firewall.

Problem description:

There was a change made to the security rule set on the firewall which unintentionally blocked incoming site-to-site VPN traffic. The problem remained unnoticed for a few hours but as soon as it was reported the configuration change was immediately rolled back to the last known good configuration (if the word “immediately” can be applicable to PA commit process at all :)). The roll back did not make any effect and the VPN tunnel remained down. Confusion was also caused by the fact that out of several tunnels only one went down and could not recover.

Findings:

Long story short at the end it appeared that there was a “stale” session in DISCARD state that, by looks of it, was constantly refreshed (and thus did not time out) as firewall was trying to reconnect the tunnel. It could not do so because of the network architecture around (only incoming VPN connections were allowed by upstream firewall) but apparently that was enough to make VPN engine ignore incoming requests from the peer it had an outgoing session to in Discard state.

The issue was resolved by resetting the “faulty”session (clear session id <xxx>) and setting the tunnel into passive mode to avoid (hopefully!) re-occurrence of the same glitch in future.

In the context of this particular issue it looked to me like the root cause was an issue/bug within VPN daemon. Though to be fair we had a similar glitch with NetFlow traffic a while ago. At some point the firewall simply stopped passing through NetFlow traffic and the cure was the same session reset which makes me think that there also might be some ongoing issues with session handling.

Modern firewalls and IPS evasion techniques

I have just had a quick glance on Internet (God bless Google) looking for reviews of evasion techniques to bypass modern firewalls. It’s amazing how vendible all these “independent” security laboratories are! The amount of s$$t some companies manage to pour onto their competitors is beyond any imagination. But as soon as you see some good results you either cannot find exact conditions and configurations used in a test or (surprise!) the winning firewall appears to be manufactured by the same company that created the testing tool and paid for the test 🙂

Though, to be fair, the Evader, being made by Stonesoft (now McAfee), is the most popular tool used to practice IPS evasion techniques.

I found only one resource clearly explaining test conditions and, as a result, the outcome of the test looked fair.

Here is a quick list of popular techniques (details are in the whitepaper referenced above).

1. Payload obfuscation and encoding (overcoming simple string matching filters)
2. Encryption and tunneling
3. Wrapping TCP sequence numbers
4. Fragmentation (splitting malicious packets into smaller fragments)
5. Protocol violations and decoy trees

So, long story short, if you have an IPS protecting a host you should not assume it’s unbreakable because of this fact and does not require any patching. If you have an unpatched vulnerability exposed to attacks, it’s sudden death is only a matter of time – easy peasy lemon squeezy with most modern IPSes. Here comes a concept of multi-layer protection – use a firewall, IPS, load balancer, web application firewall, patching, secure coding techniques and hardening altogether. Not to mention compulsory professional pen. tests (screw dumb ASV scans! :)) Then you should be good (though it’s all likely to be expensive as you may guess).

Palo Alto – assessing firewall uptime

• Management plane uptime
  

  CLI: show system resource | match up
  API: /api/?type=op&cmd=<show><system><resources></resources></system></show>&key=APIKEY

• Data plane uptime
  

  CLI: show system info | match uptime
  API: api/?type=op&cmd=<show><system><info></info></system></show>&key=APIKEY
  SNMP OID1 (uptime since host system was initialized):  .1.3.6.1.2.1.25.1.1.0 (hrSystemUptime), MIB : SNMPv2-MIB
  SNMP OID2 (uptime since snmpd was initialized): .1.3.6.1.2.1.1.3.0 (sysUpTime / sysUpTimeInstance)GUI: Dashboard > General Information

WMI probe to obtain a username of a person logged on a remote PC

Useful when troubleshooting UserID on Palo Alto (and, to be honest, in many other cases these days when NetBIOS is not such a common thing anymore):

wmic /node:<remotecomputer> computersystem get username

To perform this query you need to have permissions to read the CIMV2 namespace on the remote system.

AD base DN in group mapping on Palo Alto

I have just had some hard time configuring AD group mapping on a PA firewall. The firewall kept saying “invalid credentials” (though I had them triple-checked, re-set the password and made sure all my AD replication completed). I ended up doing a packet capture on PA which told me that in fact my context was wrong:

The solution was to go to ADUC, enable Advanced view and copy/paste object’s DN (which corrected a typo I made):