Setting up the CLI OpenStack Client (OSC) for ECP

Introduction

While several functions can be performed in the web interface, utilizing the OpenStack CLI allows advanced functionality and scripting capabilities. To begin, login to Horizon via https://cloud.cs.colorado.edu and download the OpenRC file for a specific project via
Project > API Access > Download OpenStack RC File in the Web Interface.

Objectives

Setup a Python 3 virtual environment on macOS or Windows
Installing the python-openstackclient
Decrypting Windows Passwords
Starting an instance via the command line

Videos

Setup on Windows (3:58)

Overview of how to setup the OSC within PowerShell on Windows

Setup on macOS (1:51)

Overview of how to setup the OSC within Bash on macOS

Decrypting Windows Passwords (1:24)

Process of decrypting the Windows password with the SSH private key

Video unavailable, see text guide.

Utilizing the OSC CLI (4:09)

Introduction on how to use and discover options while utilizing the OSC

See also: https://docs.openstack.org/python-openstackclient/victoria/cli/commands.html

Text Reference

Setting up the Python 3 Virtual Environment & Installing the OSC (macOS)

Launch Terminal

Install Command Line Developer Tools
Run "python3" in Terminal. If Xcode command line developer tools are missing, an install will be requested. Click Install.
If the tools are already installed, the interpreter will open instead. Enter "exit()" in this case to quit out of the interpreter.

Create a Python 3 Virtual Environment
A virtual environment is highly recommended to keep your pip3 install isolated from any other python environments that may be present. Create a new Python Virtual Environment named "osc-venv" stored in a directory "py-venvs" in the current user Document directory. To "uninstall" the isolated environment later, just delete the directory.

$ python3 -m venv ~/Documents/py-venvs/osc-venv

Activate the Virtual Environment
This environment will need to be activated every session before use. If Terminal is closed or the current session exits, it will need to be activated again.

$ source ~/Documents/python-venvs/osc-venv/bin/activate

Install the OpenStackClient (OSC CLI)

$ (osc-venv) pip3 install wheel==0.37.0
$ (osc-venv) pip3 install --upgrade pip
$ (osc-venv) pip3 install python-openstackclient==5.6.0
$ (osc-venv) pip3 install lxml==4.6.3
$ (osc-venv) pip3 install gnureadline==8.0.0

Download the OpenStack RC File from Horizon
OpenStack Horizon (WebUI) will generate a rc file based on the current project.

Login to Horizon at https://cloud.cs.colorado.edu
Switch to the desired project
Navigate to Project > API Access
On the righthand side of the page, there will be a "Download OpenStack RC File" menu
Download the file

Source the OpenStack RC File and Login via Shibboleth ECP
This example assumes the openrc file was downloaded to "Downloads" in the current home directory.
Note: If you have MFA enabled, a Duo push will occur during authentication.

(osc-venv) example:~ wazh7587$ source ~/Downloads/<downloaded openrc>.sh

--- CU Boulder OpenStack Cloud CLI OpenRC Environment --- 

Enter your organization IdP ECP URL. If you are from CU Boulder, just press enter to leave the default value.
ECP URL [https://fedauth.colorado.edu/idp/profile/SAML2/SOAP/ECP]: 

Principal: "wazh7587"
Project: "Computer Science - IT Services"
Region: "ucb-campus"
ECP URL: "https://fedauth.colorado.edu/idp/profile/SAML2/SOAP/ECP"

Your account is federated, which means python-openstackclient will need to send your credentials to your organization IdP via SAML2 ECP.

Enter password for wazh7587: 
Authenticating with IdP... Success.
Your token will expire at 2021-09-20T22:30:29+0000 (UTC).

(osc-venv) example:~ wazh7587$

Setting up the Python 3 Virtual Environment & Installing the OSC (Windows)

It is recommended to simply use bash within the Windows Subsystem for Linux (WSL) as the Linux compiler collection is much lighter than the Windows 10 SDK and C++ Build Tools.
If using Ubuntu or Debian on WSL, apt install python3-dev python3-venv gcc. Then just follow the macOS instructions above.

However, if PowerShell is desired to script against OpenStack instead of bash, then follow the instructions below.

Launch PowerShell or Windows Terminal

Install Python 3
Run "python3" in a PowerShell prompt. If it is not installed, the Microsoft Store will open. Click Install.
If python is already installed, the interpreter will open instead. Enter "exit()" in this case to quit out of the interpreter.

Install Microsoft C++ Build Tools
Some pip3 packages rely on C++ build tools
https://visualstudio.microsoft.com/visual-cpp-build-tools/

Select the "Desktop development with C++" option in the installer.

Create a Python 3 Virtual Environment
A virtual environment is highly recommended to keep your pip3 install isolated from any other python environments that maybe present.

> python3 -m venv $env:USERPROFILE\Documents\python-venvs\osc-venv

Set Execution Policy
This is required as a PowerShell script will need to be run

> Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

Activate the Virtual Environment
This environment will need to be activated every session before use. If PowerShell or Windows Terminal is closed or the current session exits, it will need to be activated again.

> Set-Location $env:USERPROFILE\Documents\python-venvs\osc-venv\Scripts\
> .\Activate.ps1

Install the Client Tools

> (osc-venv) pip3 install wheel==0.37.0
> (osc-venv) python3 -m pip install --upgrade pip
> (osc-venv) pip3 install python-openstackclient==5.6.0
> (osc-venv) pip3 install lxml==4.6.3
> (osc-venv) pip3 install pyreadline==2.1

Download the PowerShell Script

> Invoke-WebRequest -Uri "https://cloud.cs.colorado.edu/static/auth/ps-source-openrc.ps1" -outfile "$env:USERPROFILE\Downloads\ps-source-openrc.ps1"

Download the OpenStack RC File from Horizon
OpenStack Horizon (WebUI) will generate a rc file based on the current project.

Login to Horizon at https://cloud.cs.colorado.edu
Switch to the desired project
Navigate to Project > API Access
On the righthand side of the page, there will be a "Download OpenStack RC File" menu
Download the file

Translate the OpenStack RC File with the PSSource-OpenRC PowerShell Script and Login
This example assumes the openrc file is from the community project and was downloaded to "Downloads" in the current home directory.
The earlier PSSource-OpenRC.ps1 script is also assumed to be in "Downloads".
Note: If you have MFA enabled, a Duo push will occur during authentication.

Set-Location ~\Downloads
.\PSSource-OpenRC.ps1 '.\<downloaded openrc>.sh

--- CU Boulder OpenStack Cloud CLI OpenRC Environment --- 

Enter your organization IdP ECP URL. If you are from CU Boulder, just press enter to leave the default value.
ECP URL [https://fedauth.colorado.edu/idp/profile/SAML2/SOAP/ECP]: 

Principal: "wazh7587"
Project: "Computer Science - IT Services"
Region: "ucb-campus"
ECP URL: "https://fedauth.colorado.edu/idp/profile/SAML2/SOAP/ECP"

Your account is federated, which means python-openstackclient will need to send your credentials to your organization IdP via SAML2 ECP.

Enter password for wazh7587: 
Authenticating with IdP... Success.
Your token will expire at 2021-09-20T23:11:13+0000 (UTC).

(osc-venv) example:~ wazh7587$

Decrypting Windows Passwords

New windows instances will have a randomly generated password. If using Windows Server, the account name is "Administrator"; Windows 10 Enterprise will have the account name "Windows".

If the SSH key is not encrypted, the password can be decrypted in the web interface under Actions > Retrieve Password
This method is not recommended. The recommended method is to use an encrypted private key in the CLI.
Follow the below instructions to decrypt with an encrypted private key.

1. List instances and copy the instance ID

(osc-venv) example:~ wazh7587$ openstack server list
+--------------------------------------+----------------------------+
| ID                                   | Name                       |
+--------------------------------------+----------------------------+
| bae2867c-55cd-4e7c-8a69-acb40e247724 | 2k19desktop-example-server |
+--------------------------------------+----------------------------+

2. Decrypt the password

(osc-venv) example:~ wazh7587$ nova get-password bae2867c-55cd-4e7c-8a69-acb40e247724 ~/.ssh/id_rsa
Enter passphrase for /Users/wazh7587/.ssh/id_rsa:
<Windows random password will be displayed here>

3. Use the password to RDP into the instance, a password change will be prompted on first login.

Example: Starting an Instance via the Command Line

The following example will show options on how to start an instance, but the basic workflow is similar to other services.

1. Find available options

openstack --help

2. In this example, we want to work with the "server" argument to create an instance.

3. Locate help options for the desired command

openstack server --help

4. The desired option is the create argument, append --help to view its arguments

openstack server create --help

5. The usage expectations are displayed

usage: openstack server create [-h] [-f {json,shell,table,value,yaml}]
                               [-c COLUMN] [--noindent] [--prefix PREFIX]
                               [--max-width <integer>] [--fit-width]
                               [--print-empty]
                               (--image <image> | --image-property <key=value> | --volume <volume>)
                               --flavor <flavor>
                               [--security-group <security-group>]
                               [--key-name <key-name>]
                               [--property <key=value>]
                               [--file <dest-filename=source-filename>]
                               [--user-data <user-data>]
                               [--description <description>]
                               [--availability-zone <zone-name>]
                               [--host <host>]
                               [--hypervisor-hostname <hypervisor-hostname>]
                               [--boot-from-volume <volume-size>]
                               [--block-device-mapping <dev-name=mapping>]
                               [--nic <net-id=net-uuid,v4-fixed-ip=ip-addr,v6-fixed-ip=ip-addr,port-id=port-uuid,auto,none>]
                               [--network <network>] [--port <port>]
                               [--hint <key=value>]
                               [--config-drive <config-drive-volume>|True]
                               [--min <count>] [--max <count>] [--wait]
                               <server-name>

Create a new server

6. Lookup desired values

openstack image list
openstack flavor list
openstack keypair list
openstack network list
openstack security group list

7. Based on selections from above, launch an instance via the cli

openstack server create --image "Debian 10" --flavor m1.micro --key-name wazh7587 --network "projectnet3116-private (scinet-internal attachable)" --security-group icmp-restricted --security-group ssh-restricted "Debian 10 CLI Demo Instance"

8. Create and allocate a Floating IP

# Check to see if any are already allocated and available
openstack floating ip list

# Allocate a floating IP if none available
openstack floating ip create scinet-internal

9. Attach the floating IP

openstack server add floating ip "Debian 10 CLI Demo Instance" <Allocated IP>

See also: https://docs.openstack.org/python-openstackclient/victoria/cli/commands.html

Additional Resources

Additional manuals can be found in Course Modules - OpenStack

Return to Course Modules - OpenStack

Go to Next Lesson - Custom Images