Stories from the SOC is a blog series that describes recent real-world security incident investigations conducted and reported by the AT&T SOC analyst team for AT&T Managed Extended Detection and Response customers.
Executive summary
The Mirai botnet is infamous for the impact and the everlasting effect it has had on the world. Since the inception and discovery of this malware in 2016, to present day and all the permutations that have spawned as a result, cybersecurity professionals have been keeping a keen eye on this form of Command and Control (C2 or CnC) malware and associated addresses. The botnet malware utilizes malicious IP addresses that serve as intermediaries between compromised hosts and the central command server, which can use a wide range of Technique’s, Tactics, and Procedures (TTP’s) to deliver a payload in line with the malicious actor’s goals.
Recently, one of these malicious IP addresses reached out to an asset in an organization over port 22 and created an unmitigated Secure Shell (SSH) session to the company’s file server, a breach that was mitigated by the security best practices of this company preventing any follow up or lateral movement in the environment. This breach ultimately resulted in the IP getting blocklisted and stopped due to a healthy security posture that prevented malicious pivoting or exploitation.
Investigation
Initial alarm review
Indicators of Compromise (IOC)
The alarm initially came in due to an inbound connection from a known malicious IP as reported by the Open Threat Exchange (OTX) pulse related to Mirai botnet activity. OTX is open source threat sharing platform that contains a wide variety of Indicators of Compromise (IOC’s) that leverage user submitted data and the collective cybersecurity world to form an ever-evolving threat landscape.
The evidenced corresponding action ‘InboundConnectionAccepted’ is self-explanatory in that the connection was not mitigated and there was communication taking place over port 22. The associated event further detailed this inbound connection with the initiating processes, logged on user, and process parents. This revealed that the affected asset is a fileserver managed by SolarWinds software and it was likely this inbound connection was accepted in part due to typical network behavior and stateful firewall rules.
Expanded investigation
Events search
C2 activity typically utilizes positive feedback to gain persistence, relying on some sort of beacon placed in the victim’s environment that lets the attacker know there is a device or network ready for command execution. After seeing a successful connection occur with the malicious IP, the next step was to determine if the malicious IP address had further infiltrated the environment or attempted any lateral movement. A thorough search in the instance showed only the single referenced event as it pertains to the malicious IP however, the contextual events surrounding this successful connection corroborate attempted C2 activity.
Event deep dive
A further look into the event associated with the alarm shows that this is a fileserver utilizing Serv-U.exe, a File Transfer Protocol (FTP) software created by SolarWinds. The destination port 22 successfully hosted communication with the malicious IP and appears to have been automatically proxied by the software, which could also contribute to the reason this connection was accepted instead of dropped.
FTP exploits fall under the same purview as web-based attacks because of the number of public-facing file servers that exist. These allow anyone with an internet connection to potentially abuse and exploit vulnerabilities on the server. In this particular case, the public facing FTP server was open to a connection from a malicious IP and the security of the data on the asset was reliant on the subsequent security control, stressing the importance of a layered security posture with overlapping mitigative redundancies.
Reviewing for additional indicators
Immediately prior to the successful connection, there was a ProcessCreated event. That event was the utilization of Windows Defender’s ‘SenseCnCProxy.exe,’ Microsoft’s own mitigation tool for detected C2 (CnC) activity. This tool is used once more after the successful connection was made, in addition to file creations and PowerShell commands being executed.
A further look into the surrounding events showed PowerShell scripts with randomized names being created in the Temp directory of Windows, followed by the process execution of a ping command targeting internal assets.
A more detailed review of the suspicious file creations actually revealed that this was not anomalous behavior and, in fact, was part of standard operating procedure for the company. Typical internal activity that resembles malicious actions both increases the risk of noise generation with false positives, and can also increase the risk of a security event going unnoticed by virtue of being difficult to differentiate from expected activity until it is too late.
This activity needs to be heavily monitored with application allowlisting and in-depth documentation regarding the specifics of any automated action. It is a security best practice to utilize a secure, centralized management protocol for automation services that rely on secure authentication and a well documented chain of command executions. The use of automated scripts to push certain update policies was fully expected in this environment and not a byproduct of malicious actions, but this was only confirmed by the customer after the fact.
Likewise, the ping command targeting internal assets was also not anomalous and fully expected. However, this activity can easily be taken advantage of in a compromised environment, especially with regards to an FTP server typically communicating with a high volume of assets.
Response
Building the investigation
The investigation was created with the referenced events attached for internal review to ensure this activity was both legitimate and fully expected in the environment. Even though the automated script activity was not anomalous, in conjunction with the successful connection, it was still worth mentioning since it could easily be leveraged by a malicious entity and passed off as typical activity.
Mitigation for recorded C2 activity is as straightforward as blocklisting the offending IP address, however, the real concern and question is with regard to preventative measures. IP addresses are fluid and every single day, thousands of new and malicious IP addresses are introduced, making it impossible to simply blocklist all malicious IP’s and while machine learning has certainly come a long way, it has yet to be fully adopted and utilized, for good reason.
Heuristic approaches require wiggle room that sensitive, public facing assets such as file servers do not contain, which is necessary to be fully reliant on machine learning mitigative actions. In combination with a stateless firewall, however, many up-and-coming threats will not be able to find purchase via typical external scanning.
Customer interaction
Upon notification, the customer corroborated the malicious nature of the IP, verified it was unfamiliar and unexpected activity, and blocklisted the IP address. Mitigation of ongoing C2 activity is straightforward in that regard but is highly time sensitive. In this case, no malicious software was installed and there was no recorded attempt at persistence, despite the successful connection over port 22.
Limitations and opportunities
Opportunities
There is an insightful opportunity for the service provided to increase agency and provide real-time response actions, at the discretion of the customer. Utilizing AT&T’s Managed Endpoint Security (MES) platform would provide an additional barrier to malicious activity and maximize the service provided. As seen in this instance, the customer responded in a timely manner, but if that is the case, additional agency with MXDR would share a greater portion of the security load.
