Host Your Own YUM and APT Repository

Published in

ESTL Lab Notes

6 min readDec 28, 2016

Keeping servers patched and hardened is critical work. The Internet is awash with malicious exploits that could turn your server into a zombie, DDoS-ing the Internet with millions of cute cat videos, causing trillions of dollars in lost productivity. Total mayhem!

Don’t believe me? Watch Mr Robot. Reality is bleak.

Why bother setting up a repo?

If ten or more severs reside in the same environment, consider installing a local mirror to reduce the strain on your WAN link and also the hapless public mirror. Bandwidth is precious which, for obvious reasons, should be saved for production traffic and bitcoin mining.

If you said YES! to the above, you’re in the right place. This tutorial helps you set up an APT and YUM repository in an Ubuntu host in no time.

Our repo was built and tested on an Ubuntu 14.04 KVM virtual machine. All of the steps are to be performed as the root user.

Here are the major steps to in the tutorial.

Set up an APT repository
Set up a YUM repository
Schedule daily sync and sync the repos
Serve the repo’s packages
Configure the Ubuntu client
Configure the CentOS client

Set up an APT repository

This section was inspired by HowtoForge and Matrix44 (whose original URL seems down).

Begin by installing the required packages.

apt install -y apt-mirror

Create directory structure to store the repo’s packages.

mkdir -p /var/www/apt-mirror

Configure /etc/apt/mirror.list. I only synced what I needed, which is from Ubuntu’s main and universe repos and from also Singapore’s local mirror (ie. sg.archive.ubuntu.com). For your setup, add other repos to taste. As an example, I’ve included the repo for PostgreSQL.

############# config ##################
#
set base_path /var/www/apt-mirror
set nthreads 20
set _tilde 0
#
############# end config ##############deb https://sg.archive.ubuntu.com/ubuntu trusty main universe
deb https://sg.archive.ubuntu.com/ubuntu trusty-security main universe
deb https://sg.archive.ubuntu.com/ubuntu trusty-updates main universe
deb https://sg.archive.ubuntu.com/ubuntu trusty-backports main universedeb https://apt.postgresql.org/pub/repos/apt/ trusty-pgdg mainclean https://sg.archive.ubuntu.com/ubuntu

(Optional) If a proxy separates the repo from the Internet, add the following to /etc/wgetrc. apt-mirror relies on wget to download packages which in turn reads from this config file.

https_proxy = http://<your_proxy_server>:80/
http_proxy = http://<your_proxy_server>:80/
use_proxy = on

Set up a YUM repository

This segment was inspired by Deviant Engineer, CentOS Wiki and khmel.org.

Once again, start by install required packages.

apt install -y epel-release createrepo yum-utils

Create directory structure to serve the mirrored packages.

cd /var/www/yum-mirror
mkdir -p centos/7/os/x86_64/ centos/7/updates/x86_64/ centos/7/extras/x86_64/

(Optional) If a proxy separates the repo from the Internet, add this config to /etc/yum/yum.conf. Otherwise, the config file is usually works out of the box.

proxy=http://<your_proxy_server>:80/

Create the config file /etc/yum/repos.d/centos.repo. This tells reposync to sync the mentioned repos. Modify baseurl to sync from a repo close to you.

[os]
name=CentOS-7 - Base
baseurl=http://mirror.vastspace.net/centos/7/os/x86_64
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7#released updates
[updates]
name=CentOS-7 - Updates
baseurl=http://mirror.vastspace.net/centos/7/updates/x86_64
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7#additional packages that may be useful
[extras]
name=CentOS-7 - Extras
baseurl=http://mirror.vastspace.net/centos/7/extras/x86_64
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

(Optional) To add more repos, follow the instructions from the previous step, but change the config file’s filename and the config statements. Here, I also chose to sync the EPEL repo, whose config was added to /etc/yum/repos.d/epel.repo. gpgcheck=0 because EPEL has many packages with missing GPG keys. Enabling gpgcheck will result in offending packages being downloaded and deleted immediately at each sync. It’s ok to download them to the repo. What’s important is that the clients enable gpgcheck.

[epel]
name=Extra Packages for Enterprise Linux 7
baseurl=https://dl.fedoraproject.org/pub/epel/7/x86_64/
failovermethod=priority
enabled=1
gpgcheck=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7

Schedule daily sync and sync the repos

Create the file /etc/cron.daily/repo-sync to schedule a daily update of the mirror. Change --workers=4 to match the core count in your host.

# Sync Ubuntu repos
apt-mirror# Sync CentOS and EPEL repos
reposync -c /etc/yum/yum.conf -n -d -g comps.xml --download-metadata --norepopath -r os --download_path=/var/www/yum-mirror/centos/7/os/x86_64
reposync -c /etc/yum/yum.conf -n -d --download-metadata --norepopath -r updates --download_path=/var/www/yum-mirror/centos/7/updates/x86_64
reposync -c /etc/yum/yum.conf -n -d --download-metadata --norepopath -r extras --download_path=/var/www/yum-mirror/centos/7/extras/x86_64
reposync -c /etc/yum/yum.conf -n -d -g comps.xml --download-metadata --norepopath -r epel --download_path=/var/www/yum-mirror/epel/7/x86_64# Build CentOS repo metadata
createrepo --update --workers=4 /var/www/yum-mirror/centos/7/os/x86_64/
createrepo --update --workers=4 /var/www/yum-mirror/centos/7/updates/x86_64/
createrepo --update --workers=4 /var/www/yum-mirror/centos/7/extras/x86_64/
createrepo --update --workers=4 /var/www/yum-mirror/epel/7/x86_64/

Make the sync script executable.

chmod 750 /etc/cron.daily/repo-sync

Sync the repos. Before you do so, inform the network sysadmin because the process will saturate the pipe. The process will take a few light years, so kick back, relax, and have a coffee break. Dear sysadmins, this is the only opportunity in your career for a time-off.

/etc/cron.daily/repo-sync

Serve the repo’s packages

Install nginx.

apt -y install nginx

Edit /etc/nginx/sites/available/repo to configure nginx to serve the repo’s files. If you sync-ed your Ubuntu repo from a different source, replace sg.archive.ubuntu.com below with the appropriate URL. Ubuntu’s apt-mirror downloads the files to a folder named after the source URL.

server {
  listen 80;
  server_name <your-repo-hostname>;
  gzip off;
  autoindex on;
  access_log /var/log/nginx/repo-access.log;## CentOS
  location /centos {
    alias /var/www/yum-mirror/centos;
  }  location /epel {
    alias /var/www/yum-mirror/epel;
  }  location /ubuntu {
    alias /var/www/apt-mirror/mirror/sg.archive.ubuntu.com/ubuntu;
  }  location /postgresql/apt {
    alias /var/www/apt-mirror/mirror/apt.postgresql.org/pub/repos/apt;
  }
}

Enable the new nginx config and reload the daemon.

ln -s /etc/nginx/sites-available/repo /etc/nginx/sites-enabled/repo
service nginx reload

Configure the Ubuntu client

Edit /etc/apt/sources.list to configure the client to pull from your shiny new repo.

deb [arch=amd64] http://<your-repo-hostname>/ubuntu/ trusty main universe
deb [arch=amd64] http://<your-repo-hostname>/ubuntu/ trusty-updates main universe
deb [arch=amd64] http://<your-repo-hostname>/ubuntu/ trusty-backports main universe
deb [arch=amd64] http://<your-repo-hostname>/ubuntu/ trusty-security main universe

(Optional) For each additional package you want to sync, add its config to /etc/apt/sources.list.d/. Here is the config for PostgreSQL located at /etc/apt/sources.list.d/pgsql.list.

deb [arch=amd64] http://<your-repo-hostname>/postgresql/apt/ trusty-pgdg main

Sync from the new repo.

apt update

Configure the CentOS client

Configure the client to sync from your CentOS repo. Edit /etc/yum.repos.d/CentOS-Base.repo.

[base]
name=CentOS-$releasever — Base
baseurl=http://<your-repo-hostname>/centos/$releasever/os/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7#released updates
[updates]
name=CentOS-$releasever — Updates
baseurl=http://<your-repo-hostname>/centos/$releasever/updates/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7#additional packages that may be useful
[extras]
name=CentOS-$releasever — Extras
baseurl=http://<your-repo-hostname>/epel/$releasever/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

(Optional) For each additional package you want to sync, add its config to /etc/yum.repos.d/. Here is the config for EPEL located at /etc/apt/yum.repos.d/epel.list.

[epel]
name=Extra Packages for Enterprise Linux 7 — $basearch
baseurl=http://<your-repo-hostname>/epel/7/$basearch/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7

Sync from the new repo.

yum update

Wrapping Up

With luck, the configuration went as smoothly as ice cream down your throat. Like this page if it helped you!

For the cursed, like me, please proceed to the Troubleshooting room downstairs.

Troubleshooting

These are the roadblocks I encountered while configuring my repo. If you’re still stuck, leave a comment or call the Ghostbusters.

apt-mirror or reposync don’t work.
Run the commands in /etc/cron.daily/repo-sync one-by-one to debug each command. If the debugging fails, switch to an alternate upstream mirror.
apt update on the Ubuntu client results in Hash Sum mismatch.
The upstream mirror is likely corrupt so the solution is to switch to another mirror. You don’t have to re-download the entire mirror. If you followed the above steps to the letter, the packages were downloaded to /var/www/apt-mirror/mirror/sg.archive.ubuntu.com. Simply rename the folder sg.archive.ubuntu.com to the hostname of the new mirror, and re-run apt-mirror.
yum update on the CentOS client says that my GPG key is invalid.
In the CentOS client’s EPEL config, the line gpgcheck=1 enables the key verification to ensure the authenticity of the packages upstream. EPEL has a GPG key at gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7, but some repos don’t. If the repo doesn’t provide a GPG key, disable key verification with gpgcheck=0, although this is discouraged. In this case, at least sync with the upstream mirror via HTTPS.
Packages in CentOS do not update, and yum does not throw an error.
In a previous incarnation of my CentOS repo, the packages were synced to /var/www/yum-mirror/centos/7/os. The client would read from http://<your-repo-hostname>/centos/$releasever/os/Packages. For reasons beyond me, the client would install new packages but report that there were no updates. To fix this, the packages were instead synced to /var/www/yum-mirror/centos/7/os/x86_64 in the repo and the client would read from http://<your-repo-hostname>/centos/$releasever/os/$basearch.