Overview of SDC7
SmartDataCenter 7, or SDC7, is a complete cloud management solution for server and network virtualization, operations management and customer self-service. It is the software that runs the Joyent Cloud and can be used to provide, public, private, and hybrid clouds on customer premises.
This page provides an introduction to SDC7 to help get Operators started with the planning, deployment and operation of SDC7.
In this page:
- Core Services
- Instances (Virtual Machines)
This page will break down the initial concepts, terminology and architecture of SDC7. It contains almost no diagrams and is designed to be a quick read so you can move on to the detail of each topic rapidly. Links to the detailed sections are provided below where appropriate.
An SDC7 installation consists of two (2) or more servers. One server acts as the management server ("Head Node" or "HN") and the remainder are "Compute Nodes" (or "CN") which run the instances.
SDC7 is the cloud orchestration software that consists of the following components.
- An Operations Portal.
- A public API for provisioning and managing instances (virtual machines).
- A set of private APIs for use by Operators.
- Agent processes to manage communication between Head Node and Compute Nodes.
SDC7 also configures and allocates network resources to instances giving then both internal, intra-instance, and external internet connectivity.
The underlying hypervisor, SmartOS, is a powerful and lightweight container based virtualization layer that natively supports OS virtualization (containers) and hardware virtualization (KVM) for Linux, Windows, and FreeBSD.
The diagram below shows the basic architecture of SDC7.
- Head Node
- Management server.
- After initial installation it runs the Core Services, APIs, and is used to PXE Boot the Compute Nodes.
- Compute Nodes
- Hosts for instances.
- SmartOS and KVM instances.
- ZFS file system used for performance and reliability.
- Software and configuration to create an instance.
- Seed images provided by Joyent.
- Operators and end users can create images.
- Definition of resources for an instance (RAM, CPU, Disk, etc.).
- Users select an image and package to create an instance.
- Logical Networks
- SDC7 mapping of core networks.
- Each instance gets one Virtual NIC for each selected Network.
- Software deployment mechanism for SDC7.
- Defined steps to perform tasks in SDC7, such as provisioning.
- Runs a Job.
SDC7 uses a service oriented architecture. Each core service is instantiated from an image and works with other services as necessary to manage the instance, user, and networking capabilities of SDC7.
The Core Services perform a variety of roles in support of instance management. They include a single data repository (Manatee), data access/management services, APIs, and communications services as summarized below. Each core service runs in its own SmartOS instance and all of them except the Manatee data store are stateless and can operate as multiple redundant instances. (See Core Services Resilience and Continuity for more details.) The core services themselves are managed though the Services API (SAPI) and can easily be upgraded by re-provisioning from updated images.
Follow the link from each service for a detailed description:
|adminui||The Operator's Management Interface / Operations Portal|
|amon||Altering and Monitoring Service|
|amonredis||Redis data store for Amon|
|assets||Manages storage of Images on the Head and Compute nodes|
|binder||Internal DNS service for SDC7|
|cnapi||Compute Node API|
|clouadpi||The end user Public API for managing customer instances|
|dapi||Designation API, used during instance provisioning to determine which Compute Nodes are eligible for the instance|
|dhcpd||Manages IP address allocation and creation of boot images for Compute Nodes|
|manatee||High availability Postgres based data store for SDC7|
|moray||Key value data store based on Manatee. Most APIs store their data through Moray|
|rabbitmq||Inter-server message management|
|redis||Data store used for caching by data by NAPI, AdminUI and VMAPI (Data here is not persistent)|
|ufds||Unified Foundational Directory Service, an LDAP implementation built on top of Moray|
|vmapi||Virtual Machine API (management of instances)|
SDC7 provides for Role based management of infrastructure. The Account owner can create sub-users who are assigned to Roles that will allow them to perform infrastructure administration tasks on behalf of the Account holder. Through the definition of Policies allocated to Roles users can be enabled to perform such tasks as provision VM's and manage firewall rules. This capability is described in detail in User Management
Only very basic user information is stored in SDC7; this includes name, login, email, phone number and password.
Every SDC7 user must have an SSH key in their account. This key is used to authorize access to instances as described in the Instances section below. It is also used to authorize access to the publicly accessible CloudAPI.
The Internal APIs are only accessible from the Admin VLAN and are mainly used by the pre-defined workflows during the execution of jobs (provision, destroy, etc). The Operations Portal provides an interface to the internal APIs and is the recommended method of managing SDC7 and obtaining information.
However as Operators gain experience and system complexity grows it is often expedient to use the APIs for reporting, searching and summarizing information. There is also some functionality of SDC7 that is not available in the Operations Portal and has to be executed through the appropriate API.
APIs are RESTful and can be manipulated using curl commands. SDC7 also
comes with an easy to use Command Line Interface (CLI) for the APIs
that simplify their usage and syntax. Each API CLI is a command named
sdc-cnapi, etc). Details of each API can
be found by following the links in the Services list above.
The end user public API is called the CloudAPI. It is an internet accessible RESTful API authenticated using end user credentials (SSH Keys). It allows end user to create and manage their own instances using an industry standard interface. Joyent also provides a Command Line Interface (CLI) for the CloudAPI written in Node.js which can be installed on any client environment. As with the internal APIs the CLI simplifies the usage and syntax of the API.
Information on installing the CloudAPI CLI and a detailed description of the API available is available in http://apidocs.joyent.com/cloudapi/
Instances (Virtual Machines)
SmartDataCenter supports two types of instances.
- SmartOS-based instances that are Operating System virtualized and share a common kernel with the host server, thus giving bare metal performance and a low memory and disk footprint
- Guest Operating System Virtual Machines that use KVM to run Linux, Windows and FreeBSD.
Joyent SmartOS instances function as completely isolated virtual servers within the single operating system instance of the host. The technology consolidates multiple sets of application services onto one system and places each into isolated instances. This approach delivers maximum utilization, reduced cost and provides a wall of separation similar to the protections of separate instances, but on a single machine.
The SmartOS instance has been designed to be transparent to SmartOS, the underlying operating system.This visibility allows SmartOS to provide all SmartOS instances with as-needed access to a large pool of available resources, while still providing each instance with minimum guaranteed access to resources based on a pre-established fair share scheduling algorithm. In normal operating conditions, all RAM and CPU resources are fully utilized either directly by applications or for data caching. Joyent ZFS provides a write-back cache that increases I/O throughput significantly.
SmartOS supports a wide range of Unix/Linux software including:
- Languages: Node.js, PHP, Java, Python, Erlang, C, C++, Ruby, etc.
- Databases: MySQL, Oracle, PostgreSQL, Hadoop, Riak, and more.
- Middleware and Tools: memcached, New Relic, GuardTime, etc.
- Server Software: Apache, Nginx, TomCat, SAP, Glassfish, and more.
Instances of all types have one or more fixed IP addresses that are allocated at the time the instance is provisioned or when additional Network Interface Connectors (NICs) are added.
Access to instances is initially by SSH only. Users can enable
other related protocols but this is not recommended. As describe above
each user of SDC7 must have at least one SSH key added to their account
before provisioning any instances. The keys on the account are used to permit
access to instances in the following ways.
- For SmartOS instances an SDC7 technology called SmartLogin validates SSH key based access attempts directly against the key held in SDC7. Keys are not stored in the instance itself although users have the option to do this if they so chose. Thus any key stored on the SDC7 user account can be used to access a SmartOS instance even if the key is added after the instance is provisioned.
- For KVM based Linux and FreeBSD instances the SSH keys on the account are
copied to the instance at the time it is provisioned; keys added after the
instance is provisioned will not be copied down (although they can be
manually added to the instance's
SDC7 provides networking capability for instances for both internal, intra VM communication and external internet access. Networks are configured in SDC7 as Logical Networks which map to the configuration of networks in the core networking switches and routers of your organization. Networks used by SDC7 must be dedicated to SDC7 and should not have external devices or servers attached to them other than network switches and routers.
Logical Network definitions in SDC7 must match the core network configuration in every respect:
- Definition of the Subnet in CIDR format.
- VLAN ID (for tagged VLANs).
- Range of provision-able IP's.
- Gateway IP address (if applicable).
- DNS Resolver IP addresses (if applicable).
information for connectivity to other Logical
- Often required if multiple internal VLANs are configured.
When an instance is provisioned the end user selects one or more logical networks to use with their instance (up to a maximum of 32). The instance will be created with a Virtual Network Interface (VNIC) for each selected network. Each VNIC will have a static IP address. VNIC's can be added and removed from the instance at any time after provisioning.
Networks used by SDC7 are connected to the Physical NICs on the Servers from the Top of Rack Switches. The Servers do not have plumbed interfaces for these networks (apart from the admin VLAN). The VNICs created for the instances are attached to the correct physical NIC using a mechanism called NIC Tags. A NIC Tag is a simple label such as external or internal which is associated with both the physical NIC on the server and the Logical Network definition. When an instance is provisioned SDC7 will search for servers with the required NIC Tag(s) and assign the instance to one of them. As the instance is booted up its VNICs are created.
Network Pools are used to group Logical Networks together for selection during provisioning. This is typically used when there are 2 or more subnets providing the full range of IP's available to SDC7. Each of the subnets is defined as a separate Logical Network and then included in a Network Pool. End users then chose the Network Pool when creating an instance and an IP will be assigned from the first Logical Network within the pool that has an available IP address.
SDC7 provides a built in firewall capability which can be managed via
both the internal APIs and CloudAPI. End users can define a single set
of firewall rules to be applied to any or all of their instances. Rules can
be enabled, disabled and reconfigured dynamically without rebooting
an instance. Rule syntax closely follows the syntax used for
iptables as shown
in this example that allows any external system to access a specific instance
on port 80.
FROM any to vm 04128191-d2cb-43fc-a970-e4deefe970d8 ALLOW tcp port 80