Automating Virtual Machine Deployment for Research
Setting up virtual machines (VMs) interactively can be a tedious task. In this article, we’ll walk through how to automate the creation of VMs for CentOS, Debian, and Ubuntu using a simple script. This method saves time and ensure consistency across your research infrastructure.
We have developed a bash script with install_virtualization_packages function to install virtualization packages, another function setup_storage_pools to define and create storage folders where our ISO and disk files are stored, and finally deploy_virtual_machine function which automates the deployment of virtual machines with minimal manual interaction. The deploy_virtual_machine function uses virt-install for creating and managing VMs on Linux-based systems using KVM (Kernel-based Virtual Machine) and libvirt, cloud-init, preseed and kickstart files to streamline the process of configuring the VMs with specific settings.
The install_virtualization_packages Function
Before creating virtual machines, we need to ensure our host system has all the required virtualization tools installed. The install_virtualization_packages function detects the distribution and installs the appropriate packages. It sets up libvirt, KVM, and supporting utilities like virt-manager and cloud-image-utils, then restarts the virtualization service.
1install_virtualization_packages() {
2 # Define core variables
3 local pkgs service_name distro
4
5 # Detect the distro name from /etc/os-release, or exit if unavailable
6 if [ -f /etc/os-release ]; then
7 distro=$(grep -oP '(?<=^ID=).*' /etc/os-release | tr -d '"')
8 else
9 echo "Cannot detect distribution."
10 return 1
11 fi
12
13 # Install virtualization packages based on detected distro
14 case "$distro" in
15 ubuntu|debian)
16 # Core virtualization/GUI packages on Debian/Ubuntu
17 pkgs="virt-manager bridge-utils libosinfo-bin libvirt-daemon-system libvirt-clients qemu-kvm cloud-image-utils"
18 service_name="libvirtd"
19 sudo apt-get -y update
20 sudo apt-get install -y $pkgs
21 ;;
22 fedora|rhel|centos)
23 # Install Fedora/RHEL/CentOS virtualization group
24 pkgs="virt-viewer qemu-kvm libvirt libvirt-daemon"
25 service_name="libvirtd"
26 sudo dnf -y update
27 sudo dnf install -y $pkgs
28 ;;
29 arch)
30 # Install virtualization packages on Arch
31 pkgs="virt-manager libvirt qemu edk2-ovmf"
32 service_name="libvirtd"
33 sudo pacman -Sy --noconfirm $pkgs
34 ;;
35 opensuse*|suse)
36 # Install virtualization packages on openSUSE
37 pkgs="virt-manager libvirt-daemon libosinfo"
38 service_name="libvirtd"
39 sudo zypper install -y $pkgs
40 ;;
41 *)
42 echo "Unsupported distribution: $distro"
43 return 1
44 ;;
45 esac
46
47 # Add user to libvirt and kvm groups
48 sudo usermod -aG libvirt $USER
49
50 # Restart the virtualization service
51 sudo systemctl restart "$service_name"
52
53 echo "Virtualization tools installed. Service restarted. You may need to re-login for group changes."
54}
How it Works
Distribution Detection: It first detects the host's Linux distribution using /etc/os-release.
Package Installation: Based on the distribution (ubuntu/debian, fedora/rhel/centos, arch, opensuse), it defines the appropriate list of packages (virt-manager, qemu-kvm, libvirt-daemon-system, cloud-image-utils, etc.) and uses the corresponding package manager (apt-get, dnf, pacman, zypper) to install them.
User Group Membership: The function adds the current user ($USER) to the libvirt group. This is crucial for managing virtual machines as a non-root user without needing to prefix every virsh or virt-install command with sudo.
Service Management: It restarts the core virtualization service (libvirtd) to ensure all new configurations and group memberships are active. A note is provided to the user that they may need to re-login for the group changes to take full effect.
The setup_storage_pools Function
The next step is configuring storage locations for VM ISO and disk files. The setup_storage_pools function sets up persistent libvirt storage pools, creating separate directories for images and machines under a base folder (/media/$USER/CARD).
1setup_storage_pools() {
2 # Define user, group, and base directory
3 local user="$(id -un)"
4 local group="$(id -gn)"
5 local base_dir="/media/${user}/CARD"
6
7 # Fix ownership and permissions on base directory or exit if missing
8 if [[ -d "$base_dir" ]]; then
9 sudo chown -R "$user:$group" "$base_dir"
10 sudo chmod -R u+rwX "$base_dir"
11 else
12 echo "Base directory $base_dir does not exist. Is the CARD drive mounted?"
13 return 1
14 fi
15
16 # Define storage pools and paths
17 declare -A pools=(
18 ["card_images"]="${base_dir}/images"
19 ["card_machines"]="${base_dir}/machines"
20 )
21
22 # Create and activate libvirt storage pools
23 for name in "${!pools[@]}"; do
24 local path="${pools[$name]}"
25 mkdir -p "$path"
26
27 if ! sudo virsh pool-info "$name" &>/dev/null; then
28 sudo virsh pool-define-as "$name" dir --target "$path"
29 fi
30
31 if ! sudo virsh pool-info "$name" 2>/dev/null | grep -q "Active:.*yes"; then
32 sudo virsh pool-start "$name"
33 fi
34
35 sudo virsh pool-autostart "$name"
36 done
37
38 sudo virsh pool-list --all
39}
How it Works
Directory Management: It defines a base_dir (assumed to be an external drive/mount point: /media/$USER/CARD) and verifies its existence. It also fixes the ownership and permissions of this directory to ensure the current user can access it.
Pool Definitions: It uses an associative array to map desired libvirt pool names (like card_images and card_machines) to their corresponding physical directories.
Libvirt Pool Creation: For each defined pool: It creates the physical directory if it doesn't already exist. Then it uses sudo virsh pool-define-as to define a new storage pool if one with that name doesn't exist. This registers the directory with the libvirt daemon.
Pool Activation: It checks if the pool is active (Active: yes) and, if not, starts it using sudo virsh pool-start.
Autostart Configuration: It configures the pool to automatically start on host boot with sudo virsh pool-autostart.
Verification: Finally, it lists all pools to allow the user to verify the configuration. This ensures that VM ISOs and disk images are stored in locations that libvirt is aware of and can manage.
The deploy_virtual_machine Function
1deploy_virtual_machine() {
2 # Define core VM variables
3 local name iso_image disk_size ram vcpus os_variant network_name
4 local username="" password="" root_password=""
5
6 # Define paths variables
7 local base_dir="/media/$(id -un)/CARD"
8 local image_store="${base_dir}/images"
9 local machine_store="${base_dir}/machines"
10 local seed_iso
11
12 if [[ "$1" =~ ^(-h|--help)$ ]]; then
13 cat <<EOF
14Usage: $FUNCNAME -n <name> -i <iso_image> -u <username> -p <password> [-d <disk_size>] [-r <ram>] [-c <vcpus>] [-o <os_variant>]
15
16Required Flags:
17 -n, --name Name of the VM
18 -i, --iso ISO image filename in ${image_store}
19 -u, --username Username
20 -p, --password Password (prompt if not given)
21 -rp, --root-password Root Password (prompt if not given)
22
23Optional Flags:
24 -d, --disk-size Disk size (default: 20G)
25 -r, --ram RAM in MB (default: 2048)
26 -c, --cpu Number of vCPUs (default: 2)
27 -o, --os-variant OS variant string for virt-install
28 -net, --network Network name (default: default)
29 -h, --help Show this help
30EOF
31 return 0
32 fi
33
34 while [[ $# -gt 0 ]]; do
35 case $1 in
36 -n|--name) name="$2"; shift 2 ;;
37 -i|--iso) iso_image="$2"; shift 2 ;;
38 -d|--disk-size) disk_size="$2"; shift 2 ;;
39 -r|--ram) ram="$2"; shift 2 ;;
40 -c|--cpu) vcpus="$2"; shift 2 ;;
41 -u|--username) username="$2"; shift 2 ;;
42 -p|--password) password="$2"; shift 2 ;;
43 -rp|--root-password) root_password="$2"; shift 2 ;;
44 -o|--os-variant) os_variant="$2"; shift 2 ;;
45 -net|--network) network_name="$2"; shift 2 ;;
46 *) echo "Unknown option: $1"; return 1 ;;
47 esac
48 done
49
50 # Set defaults if argument is no provided
51 disk_size=${disk_size:-20G}
52 ram=${ram:-2048}
53 vcpus=${vcpus:-2}
54 network_name=${network_name:-default}
55 seed_iso="${machine_store}/${name}-seed.iso"
56
57 # Validate passed inputs
58 [[ -z "$name" || -z "$iso_image" ]] && { echo "name and iso_image are required"; $FUNCNAME -h; return 1; }
59
60 # Prompt for user password
61 while password=$(echo "$password" | xargs) && [[ -z "$password" ]]; do
62 read -s -p "Enter password for $username: " password
63 echo
64 [[ -z "$password" ]] && echo "Password cannot be empty. Please try again."
65 done
66
67 # Prompt for root password
68 while root_password=$(echo "$root_password" | xargs) && [[ -z "$root_password" ]]; do
69 read -s -p "Enter root password: " root_password
70 echo
71 [[ -z "$root_password" ]] && echo "Root password cannot be empty. Please try again."
72 done
73
74 # Ensure directories exist and define file paths
75 mkdir -p "$image_store" "$machine_store"
76
77 # Validate iso file exists
78 local iso_path="${image_store}/${iso_image}"
79 [[ ! -f "$iso_path" ]] && { echo "ISO not found at $iso_path"; return 1; }
80
81 # Auto-detect OS variant
82 detect_os_variant() {
83 local iso_file="$1"
84 local detected_os
85 detected_os=$(osinfo-detect "$iso_file" 2>/dev/null | grep -oP "(?<=Media is an installer for OS ')[^']+")
86 detected_os=${detected_os%% (*}
87 detected_os=${detected_os/ Server/}
88 detected_os=${detected_os/ Desktop/}
89 detected_os=$(echo "$detected_os" | xargs)
90
91 [[ -n "$detected_os" ]] &&
92 osinfo-query os --fields=short-id,name | tail -n +3 | awk -F'|' -v search="$detected_os" '
93 tolower($2) ~ tolower(search) { gsub(/^[ \t]+|[ \t]+$/, "", $1); print $1; exit }
94 ' || echo "generic"
95 }
96 os_variant=$(detect_os_variant "$iso_path")
97 echo "Detected OS variant: $os_variant"
98
99 # Detect OS family
100 local os_family=""
101 if [[ "$os_variant" =~ (ubuntu) ]]; then
102 os_family="ubuntu"
103 elif [[ "$os_variant" =~ (debian) ]]; then
104 os_family="debian"
105 elif [[ "$os_variant" =~ (rhel|centos|almalinux|fedora) ]]; then
106 os_family="rhel"
107 else
108 echo "Unknown OS variant, defaulting to Ubuntu-like autoinstall"
109 os_family="ubuntu"
110 fi
111
112 # Define and verify disk file or created
113 local disk_path="${machine_store}/${name}.qcow2"
114 [[ ! -f "$disk_path" ]] && {
115 sudo qemu-img create -f qcow2 -o cluster_size=2M "$disk_path" "$disk_size" || return 1
116 sudo chown libvirt-qemu:libvirt-qemu "$disk_path"
117 sudo chmod 660 "$disk_path"
118 }
119
120 # SSH key generation
121 local ssh_dir="$HOME/.ssh"
122 local ssh_key="${ssh_dir}/${name}"
123 local ssh_pub_key="${ssh_key}.pub"
124 if [[ ! -f "$ssh_key" || ! -f "$ssh_pub_key" ]]; then
125 echo "Generating SSH key for $name"
126 ssh-keygen -t ed25519 -f "$ssh_key" -N "" -C "${username}@${name}" || return 1
127 fi
128
129 # Create installation config based on OS family
130 local tmpdir
131 tmpdir=$(mktemp -d)
132 trap "rm -rf '$tmpdir'" EXIT INT TERM
133
134 # Read in the ssh key and store as a string
135 local ssh_public_key=$(<"$ssh_pub_key")
136
137 # Hash the user password for distros that use it
138 local user_passwd_hash="$(openssl passwd -6 "${password}")"
139 # local user_passwd_hash=$(mkpasswd -m sha-512 -s <<< "$password")
140
141 # Hash the root password for distros that use it
142 local root_passwd_hash="$(openssl passwd -6 -stdin <<< "${root_password}")"
143
144 case "$os_family" in
145 ubuntu)
146 local user_data="${tmpdir}/user-data"
147 local meta_data="${tmpdir}/meta-data"
148 cat > "$user_data" <<EOF
149#cloud-config
150autoinstall:
151 version: 1
152 interactive-sections: []
153
154 # Set hostname, default username, and hashed password
155 identity:
156 hostname: ${name}
157 username: ${username}
158 password: "${user_passwd_hash}"
159
160 # Install and enable SSH with the provided public key
161 ssh:
162 install-server: true
163 authorized_keys: []
164 allow-pw: yes
165
166 # Use entire disk with automatic LVM layout
167 storage:
168 layout:
169 name: lvm
170
171 # Install a small set of useful packages in addition to the base system
172 updates: all
173 packages: [vim, curl, wget, openssh-server]
174
175 # Configure language, locale, and system timezone
176 locale: en_US.UTF-8
177 timezone: Africa/Lagos
178 user-data:
179 disable_root: false
180 users:
181 - name: root
182 passwd: ${root_passwd_hash}
183 lock_passwd: false
184 runcmd:
185 - bash -c "mkdir -p /home/${username}/.ssh && echo '${ssh_public_key}' > /home/${username}/.ssh/authorized_keys && chmod 600 /home/${username}/.ssh/authorized_keys && chown -R ${username}:${username} /home/${username}/.ssh"
186 - apt-get install -y qemu-guest-agent spice-vdagent -qq
187 - systemctl enable --now ssh
188 - systemctl start qemu-guest-agent spice-vdagent
189EOF
190 cat > "$meta_data" <<EOF
191instance-id: $name
192local-hostname: $name
193EOF
194 # cloud-localds "$seed_iso" "$user_data" "$meta_data" || return 1
195 genisoimage -output "$seed_iso" -volid cidata -joliet -rock "$user_data" "$meta_data" || return 1
196 ;;
197
198 debian)
199 local debian_codename="" apt_mirror="mirror.litnet.lt" preseed="${tmpdir}/preseed.cfg"
200 if [[ "$os_variant" =~ debian([0-9]+) ]]; then
201 case "${BASH_REMATCH[1]}" in
202 13) debian_codename="trixie" ;;
203 12) debian_codename="bookworm" ;;
204 11) debian_codename="bullseye" ;;
205 10) debian_codename="buster" ;;
206 *) debian_codename="stable" ;;
207 esac
208 fi
209 echo "Using Debian Suite: $debian_suite"
210 cat > "$preseed" <<EOF
211# Language and System Selection
212d-i debian-installer/language string en
213d-i debian-installer/country string NG
214d-i debian-installer/locale string en_US.UTF-8
215
216# Keyboard Selection
217d-i console-setup/ask_detect boolean false
218d-i keyboard-configuration/xkb-keymap select us
219
220# Date and Time Selection
221d-i clock-setup/utc boolean true
222d-i time/zone string Africa/Lagos
223
224# Auto-select network interface, set hostname, and configure timezone
225d-i netcfg/choose_interface select auto
226d-i netcfg/get_hostname string ${name}
227
228# Account Setup
229d-i passwd/root-login boolean true
230d-i passwd/root-password-crypted password ${root_passwd_hash}
231d-i passwd/user-fullname string ${username}
232d-i passwd/username string ${username}
233d-i passwd/user-password-crypted password ${user_passwd_hash}
234
235# Use LVM with the atomic partitioning recipe
236d-i partman-auto/method string lvm
237d-i partman-auto/choose_recipe select atomic
238
239# Automatically remove existing LVM volumes
240d-i partman-lvm/device_remove_lvm boolean true
241d-i partman-lvm/confirm boolean true
242d-i partman-lvm/confirm_nooverwrite boolean true
243
244# Automatically remove existing RAID (md) arrays
245d-i partman-md/device_remove_md boolean true
246d-i partman-md/confirm boolean true
247
248# Finalize partitioning and allow changes to disk labels
249d-i partman-partitioning/confirm_write_new_label boolean true
250d-i partman/choose_partition select finish
251d-i partman/confirm boolean true
252d-i partman/confirm_nooverwrite boolean true
253
254# Disable CD-ROM as a source
255d-i apt-setup/disable-cdrom-entries boolean true
256
257# Enable APT mirror without specifying a country
258d-i apt-setup/use_mirror boolean true
259d-i mirror/http/mirror string deb.debian.org
260d-i mirror/http/directory string /debian
261d-i mirror/protocol string http
262
263# Enable components and services
264d-i apt-setup/services-select multiselect security, updates
265d-i apt-setup/contrib boolean true
266d-i apt-setup/non-free boolean true
267d-i apt-setup/non-free-firmware boolean true
268
269# Disable package selection
270d-i pkgsel/run_tasksel boolean false
271
272# GRUB installation
273d-i grub-installer/only_debian boolean true
274d-i grub-installer/bootdev string default
275
276# Configure sudo, install essentials, and set up SSH
277d-i preseed/late_command string echo "${username} ALL=(ALL:ALL) ALL" > /target/etc/sudoers.d/users; \
278 echo -e "deb https://${apt_mirror}/debian/ ${debian_codename} main non-free-firmware\ndeb https://${apt_mirror}/debian/ ${debian_codename}-updates main\ndeb https://${apt_mirror}/debian-security/ ${debian_codename}-security main non-free-firmware" > /target/etc/apt/sources.list; \
279 in-target apt-get update -y; \
280 in-target apt-get install -y git openssh-server spice-vdagent -qq; \
281 in-target mkdir -p /home/${username}/.ssh; \
282 echo '${ssh_public_key}' > /target/home/${username}/.ssh/authorized_keys; \
283 in-target chmod 600 /home/${username}/.ssh/authorized_keys; \
284 in-target chmod 700 /home/${username}/.ssh; \
285 in-target chown -R ${username}:${username} /home/${username}/.ssh; \
286 in-target systemctl enable sshd; \
287 in-target systemctl start sshd; \
288 in-target systemctl start spice-vdagentd
289
290# Suppresses confirmation on installation completion and enable automatic reboot
291d-i finish-install/reboot_in_progress note
292EOF
293 genisoimage -output "$seed_iso" -volid cidata -joliet -rock "$preseed" || return 1
294 ;;
295
296 rhel)
297 local kickstart="${tmpdir}/ks.cfg"
298 cat > "$kickstart" <<EOF
299#version=DEVEL
300graphical
301firstboot --disable
302
303# Language, Keyboard and Timezone
304lang en_US.UTF-8
305keyboard us
306timezone --utc Africa/Lagos
307
308# Network settings
309network --bootproto dhcp --onboot yes --activate --hostname=${name}
310
311# Security policies
312authselect select local
313selinux --permissive
314
315# Enable SSH for remote access
316firewall --enabled --ssh
317
318# Setup user accounts and passwords(system hashes the password itself)
319rootpw --allow-ssh --iscrypted ${root_passwd_hash}
320user --name=${username} --iscrypted --password=${user_passwd_hash} --groups=wheel
321
322# Wipe all existing partitions, initialize label, and use LVM for automatic partitioning.
323clearpart --all --initlabel
324autopart --type=lvm
325
326# Install required package groups and utilities
327%packages
328@^Server with GUI
329@development
330@network-tools
331curl
332wget
333openssh-server
334qemu-guest-agent
335spice-vdagent
336%end
337
338# Post-install configuration
339%post --log=/var/log/kickstart_post.log
340
341# Add SSH public key for the user
342mkdir -p /home/${username}/.ssh
343echo "$(cat "$ssh_pub_key")" > /home/${username}/.ssh/authorized_keys
344chmod 600 /home/${username}/.ssh/authorized_keys
345chmod 700 /home/${username}/.ssh
346chown -R ${username}:${username} /home/${username}/.ssh
347
348# Grant sudo privileges to the user
349echo "${username} ALL=(ALL:ALL) ALL" >> /etc/sudoers.d/${username}
350chmod 0440 /etc/sudoers.d/${username}
351
352# Enable and start SSH service
353systemctl enable sshd
354systemctl start sshd
355
356# Start guest agents (qemu/spice)
357systemctl start qemu-guest-agent
358systemctl start spice-vdagentd
359%end
360
361# Reboot after installation
362reboot
363EOF
364 # NOTE: Kickstart files are typically accessed via HTTP, but we use a CDROM
365 genisoimage -output "$seed_iso" -volid cidata -joliet -rock "$kickstart" || return 1
366 ;;
367 esac
368
369 # Initialize network
370 echo "Ensuring libvirt network '$network_name' is started and enabled..."
371 sudo virsh net-start "$network_name" 2>/dev/null || true
372 sudo virsh net-autostart "$network_name"
373
374 # Determine how to boot the installer for each distro family
375 local extra_args=""
376 local install_source=()
377
378 if [[ "$os_family" == "ubuntu" ]]; then
379 # 100%
380 install_source=(--location "$iso_path,kernel=casper/vmlinuz,initrd=casper/initrd"
381 --initrd-inject="$user_data" --initrd-inject="$meta_data")
382 extra_args="quiet autoinstall ds=nocloud\;s=/cdrom/"
383 elif [[ "$os_family" == "debian" ]]; then
384 install_source=(--location "$iso_path" --initrd-inject="$preseed")
385 extra_args="auto=true priority=critical preseed/file=/preseed.cfg"
386 elif [[ "$os_family" == "rhel" ]]; then
387 # 100%
388 install_source=(--location "$iso_path")
389 extra_args="inst.ks=cdrom:/ks.cfg"
390 fi
391
392 echo "Launching VM '$name' with OS variant '$os_variant'..."
393 # Build virt-install command dynamically
394 virt_install_cmd=(
395 sudo virt-install
396 --name "$name"
397 --memory "$ram"
398 --vcpus "$vcpus"
399 --disk path="$disk_path",format=qcow2,bus=virtio
400 --disk path="$seed_iso",device=cdrom
401 "${install_source[@]}"
402 --os-variant "$os_variant"
403 --graphics vnc
404 --network network=${network_name},model=virtio
405 --noautoconsole
406 --hvm
407 --virt-type kvm
408 --autostart
409 )
410
411 # Append --extra-args only if non-empty
412 if [[ -n "$extra_args" ]]; then
413 virt_install_cmd+=(--extra-args "$extra_args")
414 fi
415
416 # Run the command
417 "${virt_install_cmd[@]}" || { echo "VM installation failed."; return 1; }
418
419 # start virtual manager
420 virt-manager -c qemu:///system &
421
422 # Wait for shutdown, then cleanup seed ISO
423 echo "Waiting for VM '$name' to shut down before removing seed ISO..."
424 while true; do
425 state=$(sudo virsh domstate "$name" 2>/dev/null)
426 if [[ "$state" == "shut off" ]]; then
427 sudo virsh domblklist $name
428 sudo virsh detach-disk $name $seed_iso --config --persistent
429 echo "Cleaning up seed ISO: $seed_iso"
430 rm -f $seed_iso
431 sudo chown "$USER:$USER" $iso_path
432 break
433 fi
434 sleep 3
435 done
436}
The deploy_virtual_machine function is designed to automate the deployment of VMs with custom settings. It supports various options for configuring the VM, including:
Required Flags
-n,--name: The name of the VM-i,--iso: The path to the ISO image-u,--username: The username for the VM-p,--password: The password for the VM-rp,--root-password: The root password for the VM
Optional Flags
-d,--disk-size: The disk size for the VM (default: 20G)-r,--ram: The RAM for the VM (default: 2048)-c,--cpu: The number of CPUs for the VM (default: 2)
How it Works
- Define Core Variables: The function defines several variables, including the VM's name, disk size, RAM, and CPU specifications. Paths to image and machine storage directories are also specified, along with the user credentials.
- Validation and Defaults: It then validates the input parameters and sets default values for optional parameters also ensuring that the provided ISO image exists in the correct location before proceeding.
- ISO Detection and OS Variant: The function detects the OS variant from the ISO image and sets the
os_familyvariable accordingly. - Disk and SSH Key Setup: The function creates a disk image for the VM and generates SSH keys for secure access, and injects the SSH public key into the VM during installation.
- Automated Cloud-Init, Preseed or Kickstart Files: Depending on the OS variant, cloud-init files for Ubuntu, preseed configurations for Debian, or kickstart files for CentOS are generated. These configuration files automate the installation and configuration process during VM creation.
- VM Creation Using
virt-install: The script builds avirt-installcommand that dynamically adds the required options for each OS. It ensures that networking, storage, and other configurations are properly set up. - Post-Installation Cleanup
After the VM installation is complete, the script waits for the VM to shut down, cleans up temporary files, and detaches the seed ISO used for installation.