Be on the same page with us
Subscribe to our latest news
By clicking the button Subscribe you give a permission for the automatic sending of e-mails

Cutting AWS costs significantly with a free Jenkins extension

The most common problem testing teams face is the unpredictability of capacities' demand needed for a certain project, which results into ever-growing costs for capacities that may not be used to the full scale or not used at all. Depending on the size of the project, these overpayments for downtime vary from hundreds to thousands of dollars.
June 15, 2018
1
Problem description
The most common problem testing teams face is the unpredictability of capacities' demand needed for a certain project, which results into ever-growing costs for capacities that may not be used to the full scale or not used at all. Depending on the size of the project, these overpayments for downtime vary from hundreds to thousands of dollars.
2
How to solve the problem
Amazon Spot instances are significantly cheaper compared to on-demand instances (up to 90%). The only limitation users may face is Amazon guarantees 6-hour usage period after which instance may be interrupted. It depends on the demand for spot instances.
The first step is choosing the type of spot instance suitable for the project needs (note that on-demand instances are also available in conjunction with spot instances).
Spot pricing

For one of our projects, we use m3.medium instances with a single executor. They have 1 vCPU and approx. 4Gb of RAM. They are suitable for the majority of the QA/dev jobs. Moreover, it has the biggest – 90% – discount on the spot market.
Solution examples

Scalable pipeline approach integrated into AWS Spot market (no jobs – no costs) and scaling to thousands of instances when needed.

  • Customized Amazon EC2 Fleet Plugin
  • AWS Spot Fleet requests
  • Jenkins pipeline
  • QPS-infra is an integral part of the solution with a predefined Jenkins master instance. The solution unites the above-mentioned tools in one infrastructure. It is a customized solution based on a pipeline and JobDSL. Significantly cutting CI maintenance and support costs through the predefined scripts.
How does it help to cut costs?

Scalable Jenkins on demand
Jenkins is scalable on demand using Amazon Spot Fleet instances. The Spot Fleet attempts to launch the number of Spot Instances that are required to meet the target capacity that you specified in the Spot Fleet request.
Actual numbers of savings: up to 85%, it depends on the type of instance you are using.
    Is there a significant difference in cutting costs when using other types of spot instances?
    It highly depends on the tasks and the size of the project.
    *Note: we do not recommend using T2 instances for Jenkins slaves since they have limitations in performance.
    Below is a diagram showing differences in costs:
    If you are looking for professionals to ensure you achieve the above-mentioned results, they are at Solvd in SF.
    Other AWS Tools for reporting and cost optimization.