Click here to Skip to main content
13,596,314 members
Click here to Skip to main content
Add your own
alternative version

Tagged as

Stats

4.1K views
1 bookmarked
Posted 31 Dec 2017
Licenced CPOL

Deploy static web page from GitLab to Amazon S3 using Jenkins

, 2 Jan 2018
Rate this:
Please Sign up or sign in to vote.
Automatically pull static web page code from GitLab and deploy it to Amazon S3 using Jenkins.

Introduction

This article provides instructions to setup a basic deployment using Jenkins to Amazon S3 by pulling the source code from GitLab and pushing it to a S3 bucket. Deployment automation using Jenkins saves us from manually upload files to the S3 bucket using AWS online console. In addition to saving time, this method pushes the exact code that was committed in the latest Git push, therefore avoiding upload of wrong files. This is useful for static web site stored on S3 having a large number of files. 

Pre-requisite

OS: Mac OS but the main ideas are OS agnostic. 

It is assumed that you already have a static webpage on AWS S3. If not, you can check out instructions from AWS documentation: http://docs.aws.amazon.com/AmazonS3/latest/dev/HostingWebsiteOnS3Setup.html

You should also have your project added in GitLab. If not, you can have a look at their documentation: https://docs.gitlab.com/ee/gitlab-basics/create-project.html

Install Jenkins

1. Download Jenkins and install (Current LTS version as of this writing is 2.89.2): https://jenkins.io/download

2. After the installation, change the Update Site url from https to http:
Jenkins->Plugin Manager->Advanced (tab)->Update Site: http://updates.jenkins.io/update-center.json

3. Install Git Plugin (3.7.0)
Jenkins->Plugin Manager->Available (tab)
Optional. Install Green Balls plugin to make the blue success ball green. 

 

Setup to pull project from GitLab

Establish SSH authentication between your local machine and GitLab.

1. Get the Private Key Inside the File /.ssh/id_rsa and Create a Git Credential.
This site shows you how to create a Pair of Ssh Key if you do not have one: https://docs.gitlab.com/ce/ssh/README.html

2. Create a Git credential in Jenkins.

Jenkins->Credentials->System->Global credentials (unrestricted) -> Add Credentials

  1. Kind: SSH Username with private key.
  2. Select Private Key and paste the private key.
  3. Press OK

 

 

3. Add Git username and email to Jenkins Git properties.

  1. Manage Jenkins->Configure System
  2. Add your Git username and email under Git plugin section

 

4. Create a project/job in Jenkins

Jenkins->New Item

  1. Enter any name for this Job
  2. Select Freestyle Project
  3. Press Ok

 

5. Configure Source Code Management to pull the project from Git.

  1. Fill in Git project URL and select credential to use (created in step 2) under Source Code Management tab. 
  2. Press Save.

 

6. At this point you can test to see if the Jenkins job can retrieve your project from Git by clicking Build Now.

If successful, you should see a blue or green ball. You can check the logs by clicking on the blue/green ball.

To check if the project is correctly pulled from git to Jenkins workspace, click on Workspace. You should see the same files that are in your Git repository.

 

Pushing project to Amazon S3

7. Create a user to allow access to your S3 bucket.
  1. In the online AWS console->IAM (service)->Users
  2. Click Add user
  3. Enter any username you want
  4. Check Programmatic access
  5. Click Next Permission
  6. Select tab Attach Policy Directly
  7. Search for 's3'
  8. Select AmazonS3FullAccess
  9. Click Next Review
  10. Click Create user
  11. Write down the user Access Key ID and Secret Access Key as they will not be available anymore after this point.
  12. Click Close

 

8. Install and configure S3cmd (command line client to access S3)

In a terminal: 

  1. Install PIP if not already installed, to use it to install S3CMD: sudo easy_install pip
  2. Install S3cmd: sudo pip install s3cmd
  3. Configure S3cmd: s3cmd —configure
  4. Enter the Access Key ID and Secret Access Key and accept default for other questions. (Except if your region is other than US, enter a different region)
  5. Type which s3cmd to find the location of S3cmd (mine is "/usr/local/bin/s3cmd")

9. Update the Jenkins job to use S3cmd to push code to S3

  1. Return to the job created in Jenkins and click configure
  2. Go to the Build section
  3. Enter:
    PATH="/usr/local/bin:$PATH"
    s3cmd sync . s3://optimus.com --delete-removed --exclude '.git/'


    explanation: the first line tells Jenkins where to find s3cmd command by adding its location to PATH
    the second line uses s3cmd to copy all (.) to a S3 bucket name optimus.com
    --delete-removed will remove deleted files from S3
    --
    exclude '.git/' excludes the git directory to sync to S3.
  4. Click Save
  5. Click Build Now

By checking the job’s log, you should see that Jenkins pulled the code from Git and pushed it to AWS S3. 

That’s it! I hope this has helped you. 

 

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Thai Tran
Canada Canada
No Biography provided

You may also be interested in...

Pro
Pro

Comments and Discussions

 
-- There are no messages in this forum --
Permalink | Advertise | Privacy | Cookies | Terms of Use | Mobile
Web02 | 2.8.180621.3 | Last Updated 2 Jan 2018
Article Copyright 2018 by Thai Tran
Everything else Copyright © CodeProject, 1999-2018
Layout: fixed | fluid