In the days leading up to the holiday period last December, a vulnerability was identified in Citrix Application Delivery Controller (ADC) formerly known as NetScaler ADC, and Citrix Gateway formerly known as NetScaler Gateway.
The vulnerability was assigned the following CVE number: CVE-2019-19781: Vulnerability in Citrix Application Delivery Controller, Citrix Gateway and Citrix SD-WAN WANOP appliance leading to arbitrary code execution. (For the full scope of this vulnerability click here.)
It was immediately recognized that if exploited, it could allow an unauthenticated attacker to perform arbitrary code execution (in other words, your system can be hacked and your servers used for cryptomining or as a botnet for in DDoS attacks etc.)
What happened next.
While no patch was immediately available, Citrix did respond swiftly by providing appropriate mitigation advice. Then on 19 January 2020 Citrix began to release permanent fixes, which became available for all affected builds by 24 January 2020.
As a result, fixed builds have now been released across all supported versions of Citrix ADC and Citrix Gateway, and Citrix SD-WAN WANOP for the applicable appliance models. Citrix strongly recommended that customers installed these updates at the earliest possibility.
(These fixed builds can be downloaded using the following links: https://www.citrix.com/downloads/citrix-adc/, https://www.citrix.com/downloads/citrix-gateway/, and https://www.citrix.com/downloads/citrix-sd-wan/)
Why it is still a critical issue for many organisations.
Despite the mitigation advice and subsequent fixes, the timing of the vulnerability meant that many businesses were either on holiday or had imposed some form of change freeze over the holiday period.
As a result, in January the National Cyber Security Centre (NCSS) announced that it was investigating multiple exploitations of this critical vulnerability, with attackers deploying various payloads once exploitation had taken place.
Threatpost, the leading independent news site that reports IT and business security, claimed that over 25,000 servers across the world were vulnerable. It also highlighted the severity of the vulnerability in a post, stating that it packs a double punch, as not only is this vulnerability easy to take advantage of, once exploited remote attackers could access private network resources without requiring authentication.
Furthermore, FireEye published a blog post revealing threat activity, including repeated exploitation attempts in the travel, legal, financial, and education sectors. A few days later, FireEye released another blog post detailing the activities of a threat actor gaining access to vulnerable devices, cleaning up known malware, and deploying a previously unseen payload to block follow-up exploitation attempts. However, this payload – now referred to as NOTROBIN – also serves as a backdoor.
How to ensure your system hasn’t been compromised.
To help identify compromised systems associated with CVE-2019-19781, FireEye and Citrix worked closely together to release a tool that searches for indicators of compromise (IoC) associated with attacker activity. (This tool is freely accessible in both the Citrix and FireEye GitHub repositories.)
The free tool is designed to allow you to run it locally on your Citrix instances and receive a rapid assessment of any potential indications of compromise in the system based on known attacks and exploits.
In addition to applying the previously released mitigation steps and installing the permanent updates, Citrix and FireEye strongly recommend that all Citrix customers run this tool immediately in order to increase their overall level of awareness of potential compromise, and take the appropriate steps to protect themselves.
Note: If you do detect any suspected exploitation, you should report it to the NCSC via the website.
We ran the tool and found compromises.
Any compromises should first be reported to the NCSC, then take the following steps:
- Assess the compromises picked up in the report and tackle each issue. Some of the items picked up by the scan tool may be a false positive, e.g. the tool picks up and reports on all .xml files listed in /var/vpn/bookmark/some of bookmark .xml files may be legitimate.
- Remediate and re-scan.
- Monitor for unusual activity.
- Migrate to a new appliance.
Still unsure about what to do? Don’t worry, help is at hand.
If you have any doubts or worries about any aspect of this widespread vulnerability in Citrix Application Delivery Controller, Citrix Gateway or Citrix SD-WAN WANOP appliance, don’t hesitate to contact us.
For immediate help and advice contact Peter Grayson on 0161 537 4980 or email peter.grayson@quadris.co.uk
