Welcome to our blog.
Preventing fallout from your CI/CD platform being hacked
Preventing cloud takeover after the compromise of credentials
Continuous integration and continuous delivery/deployment (CI/CD) tools are no longer a luxury for any startup. The fastest-moving startups have learned that shipping big, ambitious ideas works best by shipping tiny, incremental and easy-to-review changes. The most productive among them ship 40 times a day. Some even up to 80 times per day. This can only be done safely by leveraging a CI/CD tool such as CircleCI, GitHub Actions, and GitLab’s pipelines to name a few.
CI/CD attracts hackers
A lot of startups and bigger companies are using these tools nowadays. In order to have them deploy code to your cloud, you have to store special API secrets inside them. That makes CI/CD tools high-value targets for hackers. In fact, they have a history of being hacked all the time.
Have a look at these incidents, which are just some of the recent breaches that have been publicly disclosed:
- CodeShip: “Critical Security Notification: GitHub breach” (2020)
- GitHub: “Exploiting GitHub Actions on open source projects” (2022)
- GitLab: “Action we've taken in response to a potential Okta breach” (2022)
- Jenkins: “A critical Jenkins bug discovered” (2020)
As you can see, it happens pretty regularly. How are you defending yours?
How do I defend my cloud infrastructure against breaches like these?
When one of these CI/CD platforms gets hacked, they will usually disclose the breach. That tends to happen within a day of them noticing the breach. However, a breach could be active for weeks before they find out. Unfortunately, that time can be used to escalate access to all customers of the platforms.
Aikido helps you identify your CI/CD defenses
Luckily, there are some methods to make sure you stay safe even if your platform of choice gets hacked. Aikido Security’s new integration with AWS will alert you if your cloud does not actively take any of the following measures. Use our free trial account to see if your cloud already has defenses against these threads.
Steps to take to defend your CI/CD:
- When assigning IAM roles to your CI/CD platform, make sure they are restricted by IP. Most CI/CD tools have an option to only send traffic from a specific set of IP addresses. That option renders stolen API tokens unusable outside of the CI/CD infrastructure. A hacker won’t be able to use them on their own servers, which should slow them down a lot and potentially block them altogether.
- When creating credentials for CI/CD platforms, spend time crafting minimal access. Don’t give out admin rights.
- Don’t put all your eggs in one basket: split your clouds up into multiple accounts. This minimizes the effect of a breach. For example, a breach of your staging environment credentials should not result in a breach of your production environment.
- Use single sign-on (SSO) or multi-factor authentication (MFA). A no-brainer really.
Sadly (but realistically), you should assume your CI/CD will get hacked one day. So when the time comes, make sure to rotate all deployment tokens asap.
How to Close Deals Faster with a Security Assessment Report
In today's crazy competitive business world, startups face a bunch of challenges when it comes to sealing the deal. One of the most significant is building trust with potential leads. Especially around security. Establishing trust with potential customers can be critical for a startup's success.
That’s why we’re launching a new feature to help solve this issue: Security Assessment Reports
When a startup shares a comprehensive security assessment report, they show they mean business when it comes to security. They’ll build trust quickly and speed up closing the deal.
In this blog post, we explore how our reporting feature works. We also look at why startups need to communicate trust from the start and how that leads to winning more business.
What’s in this Security Assessment Report?
Let's face it, trust is all about being open and honest. Aikido’s security assessment report spills the beans! Customers receive information on the startup's security practices, OWASP Top 10 score, and vulnerabilities. They can even learn how fast the startup handles risks. The willingness to share this info enables startups to prove they're dead serious about keeping everyone's data safe and sound.
Tailored Approval Flow
Not everything needs to be an open book, right? That’s why our security assessment report lets startups customize exactly which information they’d like to share. This way you’re able to share only what the right leads really need. It's like offering a sneak peek without giving away the whole shebang. Startups keep control over sensitive info. Meanwhile, they give leads the reassurance needed to move forward.
Unbiased Standards & Best Practices
Our security assessment report follows top-notch standards and best practices. Consequently, this boosts the startup's credibility and professionalism.
ISO 27001:2022
ISO/IEC 27001 is an international information security management system standard. It provides a list of compliance requirements against which organizations and professionals can be certified. Additionally, it helps organizations establish, implement, maintain, and improve an information security management system (ISMS). Aikido analyzes all items related to code & cloud security & automates monitoring.
SOC 2
SOC 2, aka Service Organization Control Type 2, is a cybersecurity compliance framework developed by the American Institute of Certified Public Accountants (AICPA). The primary purpose of SOC 2 is to ensure that third-party service providers store and process client data in a secure manner. Just like for ISO 27001, Aikido analyzes all items related to code & cloud security & automates monitoring for you.
OWASP Top 10
OWASP Top 10 is globally recognized by developers as the first step towards more secure coding. It’s a listing of the most important security risks web applications face. When you fix the security issues in the OWASP Top 10 list then you can be assured you’ve dramatically improved your application’s security. Moreover, Aikido gives you an instant view of which OWASP Top 10 security issues you need to solve.
Aikidosec Benchmark
At Aikido we’ve built our own Aikidosec benchmark, which scores your environment compared to other Aikido users. In your security assessment report, you’re free to share this benchmark with your customers to show them you’re among the top X% security performant startups.
“Secured with Aikido” - A Badge of Trust
We've added an exciting bonus feature. To help our users show their commitment to security, we’ve created a special badge for your website. The badge makes it easy for customers to request a security assessment report with just a few clicks. The badge serves as an external, unbiased validation, providing assurance to your customers that you are implementing security measures.
Getting Ahead of the Competition
In a crowded marketplace, standing out from the crowd is essential. We believe the security assessment report gives startups a competitive edge by showcasing their commitment to security. How can startups position themselves as trustworthy partners of choice? Our advice is to address concerns in advance through a robust, proactive and transparent approach to data protection.
Closing Deals Faster and Growing Revenue
Let's cut to the chase, shall we? Startups want to close deals ASAP. Additionally, they want to grow their revenue. Security assessment reports help build trust and credibility allowing startups to speed up their sales cycle. This means less time wasted, fewer resources spent, and more contracts signed.
Try it out now by requesting Aikido’s own security assessment report.
Want a report for your startup?
Go into the app and generate your own report.
Automate Technical Vulnerability Management [SOC 2]
How to become compliant without imposing a heavy workload on your dev team
Achieving compliance with ISO 27001 and SOC 2 can be a daunting task, especially when it comes to technical vulnerability management. However, with the right tools and support, it doesn't have to be. In this blog post, we'll discuss how Aikido and Vanta can help you tackle the technical aspects of SOC 2 compliance.
Covering the Technical Vulnerability Management Requirements for SOC 2
To achieve compliance with SOC 2, companies need to implement technical vulnerability management measures. This involves identifying, prioritizing, and addressing vulnerabilities in your codebase and infrastructure. To cover these requirements and ensure your systems are secure, you need to follow a series of steps and implement a process:
- Conducting a risk assessment
The first step is to conduct a risk assessment of your codebase and infrastructure to identify potential vulnerabilities. This involves analyzing your systems and identifying potential weaknesses that could be exploited by attackers. - Prioritizing vulnerabilities
Once you've identified potential vulnerabilities, you need to prioritize them based on their severity and potential impact on your systems. This will help you to focus your efforts on addressing the most critical vulnerabilities first. - Addressing vulnerabilities
The next step is to address the identified vulnerabilities. This can involve implementing patches, upgrading software, or making configuration changes to your systems. - Testing for effectiveness
After addressing the vulnerabilities, it's essential to test the effectiveness of the fixes you've implemented. This involves conducting penetration testing and other security tests to ensure that your systems are secure. Pentests are not a hard requirement for SOC 2 though. - Ongoing monitoring
Finally, it's essential to continually monitor your systems for potential vulnerabilities and threats. This involves implementing a vulnerability management program that regularly scans your codebase and infrastructure for potential vulnerabilities and risks.
By following these steps, companies can ensure that they meet the technical vulnerability management requirements for SOC 2 compliance and have secure systems in place to protect their data and infrastructure.
Automating the process with Aikido
To become compliant, you can implement the process manually or use a vulnerability management platform, such as Aikido. We’ll run you through the process and how to automate it.
1. Conducting a risk assessment
By plugging into your code and cloud infrastructure, Aikido automatically conducts a risk assessment. It thoroughly analyzes your systems, identifying potential vulnerabilities that could be exploited by attackers. As Aikido is agentless, you can get a full overview in 30 seconds. No more hours wasted installing expensive software or configuring and maintaining free open-source tools.
2. Prioritizing vulnerabilities
Once the risk assessment is complete, Aikido prioritizes the vulnerabilities. Instead of overwhelming you with a long list of all the vulnerabilities present in your system. Vulnerabilities are deduplicated and auto-triaged, you’ll only see the ones that truly matter and are exploitable. This way, you can focus your efforts on addressing the most critical vulnerabilities first.
3. Addressing vulnerabilities
Addressing vulnerabilities can be a manual task, but Aikido makes it easy. Features such as autofix allow you to make a PR with one click. Next to that, Aikido integrates fully with the tools you’re already using. Whether it's implementing patches, upgrading software, or making configuration changes.
4. Testing for effectiveness
To ensure the effectiveness of the fixes implemented, we advise doing a pentest. This way, you can validate the effectiveness of the security measures and ensure that your systems are robust against potential attacks. Though, for SOC 2, this is not required. Aikido typically works with Shift Left Security, but you’re free to pick any consultant you’d like.
5. Ongoing monitoring
Additionally, Aikido helps you with ongoing monitoring, a crucial aspect of maintaining secure systems. Aikido scans your environment every 24 hours for any new vulnerabilities and risks. By continuously monitoring your systems, you can stay proactive in identifying and addressing any emerging vulnerabilities or threats.
With Aikido, you can automate the entire process of vulnerability management, from risk assessment to vulnerability prioritization, addressing vulnerabilities, testing for effectiveness, and ongoing monitoring. By leveraging Aikido's capabilities, companies can meet the technical vulnerability management requirements for SOC 2 compliance and establish a secure environment to safeguard their data and infrastructure.
Why integrating Aikido & Vanta will save you time & money
No more manual processes to follow up on
Aikido puts technical vulnerability management on autopilot. The platform continuously monitors your security posture in the background. You’ll only be notified when it’s actually important. On top of that, it automates 16 Vanta tests & helps pass 5 Vanta controls.
No more time wasted triaging false positives
The majority of security platforms indiscriminately send all identified vulnerabilities to Vanta. This results in a significant waste of time as you have to sift through numerous false positives. For example, when you use other security tooling, all the vulnerabilities found are sent to Vanta, which means you have to spend a lot of time sorting through them. On the other hand, Aikido has built an auto-triaging engine that acts as a helpful filter, saving you precious time.
No more wasted money on expensive licenses
The security industry is plagued by predatory pricing models that are overly complex. Some companies adopt user-based pricing, which encourages developers to share accounts, ultimately compromising security. Others opt for code-line-based pricing models, which get expensive very quickly. However, we reject these approaches and instead offer a straightforward fixed fee pricing per organization. With Aikido, you can begin at just €249 per month. By choosing our model, you can expect to save approximately 50% compared to competitors.
Vanta, an essential piece of the puzzle
To implement SOC 2, you need to do more than just technical vulnerability management. You’ll need a general, overall Security Compliance Software solution. A platform such as Vanta automates 90% of the complex and time-consuming process of SOC 2. And on top of that, it integrates seamlessly with Aikido. Making all aspects of technical vulnerability management dead simple.
Why wait? Try Aikido today for free (onboarding takes 30 seconds) and fast-track your SOC 2 compliance.
Preventing prototype pollution in your repository
If you arrived on this page directly from an Aikido Autofix Pull Request and just want to learn how to finish the PR, skip to ‘How to implement’.
The JavaScript ecosystem has a problem and it’s called prototype pollution. For a SaaS company building with JavaScript/npm, typically up to 20-30% of all known vulnerabilities (CVE) detected in the dependencies are caused by prototype pollution. They are usually not easy to exploit, but in bad cases, they can lead to remote code execution flaws. That means they are hard to ignore altogether.
Two ways to prevent prototype pollution
There is experimental support in Node.js to freeze the prototype by default under a flag called --frozen-intrinsics. Enabling that flag defeats all prototype pollution attacks. However, we can’t recommend it yet because it’s experimental and it also won’t work on frontend codebases. That would prevent you from building the same protection across your frontend and backend.
An alternative is nopp (No Prototype Pollution), a 14-line library that freezes the prototype (and some other entry points).
How to implement
1. Importing the library
After you’ve installed nopp, all you have to do is go to the script that starts your app. There you simply require the library, after you’ve required all your other libraries. Example commit below:
2. App-wide safety check for libraries depending on prototype manipulation
The reason we’re freezing the prototype after including other libraries is that some of your other libraries may rely on changing the prototype to work! Even after making sure to freeze the prototype AFTER including other libraries, it’s still possible you will need some refactoring before your app works again!
Case in point, Amazon’s own aws-sdk for Node.js makes changes to the prototype during the construction of objects such as “new AWS.S3(..)”. In such cases, you might have to do a refactor as shown below, making sure the object is created when your Node.js process is booted and not during a later phase.
Making sure your app still works after this change might be a bigger time investment for larger repositories that have low test coverage. For smaller repositories, it’ll be worth it to never have to deal with prototype pollution again. For bigger repositories, this might be more complex, but the engineering investment will likely still have a positive ROI over the longer term.
How does a SaaS startup CTO balance development speed and security?
Learnings from past SaaS companies
In a typical SaaS startup you’ll find the dev team under heavy pressure to deliver new features. You usually have competitors who might be better funded, you’ll have the sales team asking for one last feature to close the deal, the customer support team asking for a bugfix. It can be hard to prioritize security unless a bigger enterprise customer demands it or your board imposes it from the top.
In most startups, you might not have a full range of tech profiles at your disposal: you might not have a full-time product manager yet, you probably don’t have a full-time head of security on your development team. A dev team under pressure to deliver will be forced to cut a lot of corners, even when it comes to security.
I have been the CTO of 3 early-stage SaaS startups. (Teamleader, Officient & Futureproofed) Below I’ve outlined my learnings from those past SaaS company experiences.
Avoid reinventing the wheel
A great example: don’t build a login system with passwords. Yes, you could build it in a few days, but the cost to maintain it and add advanced protection features in the future will be very high. Consider using a login via Single-sign on such as Gmail or use a service such as Auth0.com. You’ll not only have a smoother, more feature-rich login experience, but you also won’t have to worry about a whole class of login-related security aspects (brute-forcing, leaked user credentials in 3rd party services, avoiding and detecting account takeover attacks, validating email addresses, logging...).
In the end, if you pick the right vendor, you’re not just reducing your attack surface, you’re also winning time that can be spent on more valuable features.
Other great examples that could save you weeks or months are:
- Don’t store credit cards, use something like Stripe or Mollie or Chargebee
- Don’t run complex software like MySQL or PostgreSQL or ElasticSearch yourself. Use managed cloud services such as RDS.
- Don’t build your own logging systems. Use systems like Sentry or Papertrail (and don’t log any sensitive data there)
- Don’t run your own email (SMTP) servers, use a managed service like Sendgrid or Mailgun
- Don’t build websocket servers, use managed services such as Pusher.com
- Don’t build your own featureflagging system, use a service such as Launchdarkly
- Don’t build your own calendar integration, use a tool like cronofy.com
- When building integrations in general, check for Unified APIs in that space such as Apideck.
Invest some time in a crisis communication setup
Make sure you have tools set up such as a chat application to talk to your customers, or better, a managed status page or Twitter account where you can communicate problems. In case something bad happens it’s a great practice to enable the rest of your company to communicate to your customers while you focus on fixing the problem during a crisis.
Add global guardrails
You’re probably reviewing Pull Requests from your developers, great! It’s quite a task to review them for maintainability, performance, functionality. Do you have time to also review them for security? For sure you’ll be able to cover some risks, but it’s nice to be able to sideline some risks by adding some guardrails and global configuration.
Sometimes, you can get lucky and global fixes exist for specific issues.
- If you use nodeJS and don’t like thinking about prototype pollution, you should freeze the prototype to disable this class of attacks app-wide. Aikido can automatically build a Pull Request that does this for you.
- If you use SQL and are afraid of SQL injection attacks (you should be), you can use a WAF (like AWS WAF) or RASP (like Datadog’s app security) for app-wide protection
- If you’re discovering a lot of XSS attacks, you can likely remove a large part of them by introducing a very strict CSP policy in the browser.
- If you’re doing a lot of outbound requests based on customer input, you might be vulnerable to SSRF attacks. While it might be hard to block this completely, you might mitigate damages by making sure it can’t lead to something worse (such as an IMDSv1 attack on IAM credentials in AWS). Aikido monitors this in your AWS cloud by default.
- When dealing with object storage, avoiding specific types of functions such as Pickle, XML, (un)serialize in Java and PHP,.. and instead going for simple storage options such as JSON can avoid a lot of typical exploits related to unsafe unserialization. Aikido monitors for these kinds of functions in your codebase by default.
- If you use a CDN or caching mechanisms, use separate domains for your static assets with separate CDN configurations to avoid all kinds of cache poisoning attacks (ChatGPT ran into this trap recently)
Educate your developers with this one simple trick
There’s an easy hack (pun intended) you can implement in your processes. One quick question to ask the dev team during sprint planning:
How can the functionality we’re building get hacked? (& how can we prevent that from happening)
While the dev team may not anticipate every potential abuse scenario, this methodology is dead simple to scale. It’s a small extra step that encourages developers check in on security considerations before undertaking any development work.
Assign priorities to different parts of your codebase
Not everything will be as big of a target for hackers. Key components they love to target are :
- Payment systems, wallets, credit systems
- Functionality that connects to expensive APIs like SMS, voip (think Twilio)
- Password reset, login systems, user invites
- Export features such as PDF, Excel,.. exports that might perform risky operations
- Anything related to file, image uploads
- Features that do outbound requests by design, such as webhooks
- Any kind of feature that sends email, especially with personalized content
PS: Aikido can help you focus on only the top repositories in your startup’s universe by assigning risks levels to each codebase.
Don’t forget about the human aspect
As CTO in a small startup, you’re usually also the one in charge of operational security. Provide your team with the means to secure their accounts. Make sure they use hardware Yubikeys or software apps for 2FA and if possible, enforce it. Tools like Gmail allow enforcing this. It’s a great first line of defense against phishing attacks.
Staying up-to-date with security practices is hard
Learning about new kinds of attacks on software is hard. It’s hard enough to follow up on updates to the languages that you use (Python, Node, Go,..) or OS patching or popular exploits in open-source packages. Specific attack become more popular over time so it pays off to follow the trends. For example, after noticing an uptick in subdomain takeover attacks last year, Aikido introduced a subdomain takeover monitoring tool to prevent that risk and to automate the practice of monitoring these DNS records.
Automate away
At my previous companies, we would try to get to a next level of security by having a security person build a calendar with recurring security tasks. Do an access-review for all apps every quarter. Do a leaked-secrets scan on the code every couple of week, clean up the open source package CVEs every Friday, do a SAST scan every so often, verify the DNS records for subdomain takeovers every month,... The output of these tasks was hard to triage and make actionable for developers. Worse, when this person left the company it was hard for anyone else to pick up these tasks because they required a lot of specific knowledge.
This is where the idea of Aikido started growing. We needed a solution to automates all of the above, filter out the noise and get the tasks in front of your developers in Slack or Jira.
Start scanning your code & cloud with Aikido today. Sign up over here and start scanning for free.
How a startup’s cloud got taken over by a simple form that sends emails
Preventing total cloud takeover via SSRF attacks
This is the story of an attacker who gained access to a startup’s Amazon S3 buckets, environment variables, and various internal API secrets, all via a simple form that sends an email. Even though this is a true story, I’m keeping the startup’s name a secret.
How they got in
It all starts with a PHP application sending out an email. To load some of the fancy images to the email as attachments, the application needs to download them. In PHP this is easy. You use the function file_get_contents and that is where the fun begins.
Of course, some of the user input for this email was not fully checked or HTML-encoded, and thus a user could include an image such as <img src=’evil.com’/>
. Now, in theory, this is not that too bad, but sadly this PHP function is very powerful and can do much more than load images over the internet. It can also read local files and more importantly: files over the local network instead of the Internet.
Instead of evil.com, the attacker entered a special local URL. You can use this URL to get the IAM credentials linked to the role of the AWS EC2 server you're running with a simple GET request.
<img src=’http://169.254.169.254/latest/meta-data/'>
The result was that the attacker got an email that included the IAM credentials for the EC2 server in an attachment in the mailbox. Those keys give the attacker the ability to impersonate that server when talking to various AWS services. It all goes downhill from there...
Why is this even possible in the first place?
The system that loads these IAM keys is called IMDSv1 and Amazon released a new version in 2019 called IMDSv2. With IMDSv2, you need to send a PUT request with a special header to get your IAM credentials. That means a simple GET-based URL loading function like file_get_contents can no longer cause that much damage.
It’s unclear what the adoption of IMDSv2 is as of 2023, but it’s clear Amazon is still taking measures to increase its adoption and we’re seeing IMDSv1 still being used in the wild.
The compromise of the IAM keys leads to further compromises: The S3 buckets could be listed and their contents read. To make matters worse, one of the S3 buckets contained a cloudformation template, which contained sensitive environment variables (eg Sendgrid API keys).
How do I defend my cloud infrastructure against this?
Now, what could be done to prevent this total loss of data? Your developers could be extra careful and take care to use an allowlist for the URLs they pass on to file_get_contents. They could even verify that the content they receive is an image if they are expecting an image. The reality is however that these kinds of mistakes are hard to avoid as a developer.
It would be a lot better to ensure your infrastructure has extra defenses against these attacks. Our new integration with AWS inside of Aikido security will alert you if your cloud does not actively take any of the following measures. Each of these measures on its own would have stopped this particular attack, but we recommend implementing all of them. Use our free trial account to see if your cloud is already defended against these threats. See how Aikdido protects your app against vulnerabilities here.
Steps to take:
- Migrate your existing IMDSv1 EC2 nodes to use IMDSv2
- Don’t store any secrets at all in the environment of your webservers or in cloudformation templates. Use a tool such as AWS Secrets Manager to load the secrets at run-time.
- When assigning IAM roles to your EC2 servers, make sure they have extra side conditions such as restricting them to be usable only from within your local AWS Network (your VPC). The example below allows your server to read from S3, but only if the EC2 server is communicating via a specific VPC endpoint. That’s only possible from within your network, so the attacker wouldn’t have been able to get to the S3 buckets from his local machine.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "rule-example",
"Effect": "Allow",
"Action": "s3:getObject",
"Resource": "arn:aws:s3:::bucketname/*",
"Condition": {
"StringEquals": {
"aws:SourceVpce": "vpce-1s0d54f8e1f5e4fe"
}
}
}
]
}
About “The Kill Chain”
The Kill Chain is a series of real-life stories of attackers getting to the crown jewels of software companies by chaining multiple vulnerabilities. Written by Willem Delbare, leveraging his ten years of experience in building & supporting SaaS startups as a CTO. The tales come directly from Willem’s network and all really happened.