IBM PureFlex SmartCloud Entry Solution Technical Walkthrough

(1)

DTU – Distributed Systems

IBM PureFlex SmartCloud Entry Solution Technical Walkthrough

Lyngby

April 18

^th

, 2013

Ib F. Kristensen, ibfk@dk.ibm.com

IT Architect

Nordic PureFlex Product Manager

(2)

Agenda

IBM PureFlex SmartCloud Entry Solution

IBM

Concept overview of the system

Design issues/requirement for the Flex Chassis The physics of the Flex Chassis

The networking in the Flex Chassis

The Compute power in the Flex Chassis Management of the hardware

The Cloud software stack

Wrap up

(3)

IBM

(4)

IBM Corporation

* source: ibm.com/investor/4q12/presentation/4q12.pdf

Key figures

Headquarters

Armonk, New York, USA CEO

Virginia M. Rometty Employees 2012 434,246

Revenue 2012

104.5 billion US-Dollar Homepage

ibm.com

With revenues of 104.5 billion US-Dollar in 2012, IBM is one of the world’s largest provider of information

technology (hardware, software and services) and B2B solutions. The company has more than 430,000

employees worldwide and operates in more than 170 countries.

For more financial information please visit ibm.com/investor/financials/index.phtml

IBM revenue by industry* (in billion USD)

IBM revenue

Revenue in million USD 2012 2011 2010

Global Services

Global Technology Services Global Business Services

58,802 40,236 18,566

60,163 40,879 19,284

56,424 38,201 18,223

Hardware 17,667 18,985 17,973

Software 25,448 24,944 22,485

Global Financing 2,013 2,102 2,238

Other 577 722 750

Total revenue 104,507 106,916 99,870

Other: 5.9 Small and medium

business: 24.2

Communications: 9.5

Distribution: 9.2

Industrial: 9.7

Financial services: 30.7

Public: 15.3 Total revenue

104.5 billion USD

(5)

5

Introduction to the “New” IBM

(6)

Concept overview of the system

(7)

What is Cloud Computing?

A user experience and a business model

Rapidly provisioned standardized offerings Usage based pricing

An infrastructure management and services delivery methodology

Virtualized resources with elastic scaling

Monitor & Manage Services & Resources

Cloud

Administrator

Datacenter Infrastructure

Service Catalog, Component Library

Service Consumers

Component Vendors/

Software Publishers

Publish & Update Components, Service Templates

IT Cloud

Access Services

(8)

The IBM CCRA (Cloud Computing Reference Architecture)

defines 3 different roles: the Cloud Service Consumer, the Cloud Service Provider and the Cloud Service Creator.

Governance

Security, Resiliency, Performance & Consumability

Cloud Service Creator Cloud Service

Consumer Cloud Service Provider

Common Cloud

Management Platform (CCMP)

Operational Support Services (OSS) Cloud Services

Inf rastructure-as-a-Service Platf orm-as-a-Service

Sof tware-as-a-Service Business-Process-

as-a-Service

Business Support Services (BSS) Cloud

Service Integration

Tools

Consumer In-house IT

Service Creation Tools

Inf rastructure Existing & 3^rdparty

services, Partner Ecosystems

(9)

Infrastructure-as-a-Service Platform-as-a-Service Application-as-a-Service

Servers Networking Storage

Middleware

Collaboration

Financials

CRM/ERP/HR Industry

Applications

Data Center Fabric Shared virtualized, dynamic provisioning

Database

Web 2.0 Application Runtime

Java Runtime Development

Tooling

Four major categories of Cloud Computing services are

emerging

^Examples

Business Process-as-a-Service

Employee Benefits Mgmt.

Industry-specific Processes

Procurement

Business Travel

(10)

The IBM CCRA (Cloud Computing Reference Architecture)

defines 3 different roles: the Cloud Service Consumer, the Cloud Service Provider and the Cloud Service Creator.

Governance

Security, Resiliency, Performance & Consumability

Cloud Service Creator Cloud Service

Consumer Cloud Service Provider

Common Cloud

Management Platform (CCMP)

Operational Support Services (OSS) Cloud Services

Inf rastructure-as-a-Service Platf orm-as-a-Service

Sof tware-as-a-Service Business-Process-

as-a-Service

Business Support Services (BSS) Cloud

Service Integration

Tools

Consumer In-house IT

Service Creation Tools

Inf rastructure Existing & 3^rdparty

services, Partner Ecosystems

(11)

High-level view of IBM SmartCloud Entry for software cloud solution

(12)

IBM Flex System Enterprise Chassis

Support technology advancements through the next several years – Future processors, memory, I/O, storage, specialty hardware – Faster internal bus speeds

– Faster I/O speeds

Energy efficient cooling and power system – High efficiency components

• Sophisticated control systems

• Compatible with future, high efficiency data center infrastructure – New Compute and Storage nodes (Leadership Intel Performance) – New scalable switch elements and IO adapters

– New integrated management node and chassis management elements

• Ability to manage multiple chassis

(13)

Design issues/requierment for

the Flex Chassis

(14)

Some of the design issues/requirement for the Flex System

Based on standard architecture and components (x86, Power, FC, ethernet...) Lifetime of appx. 10 years

Low cost Simplicity Flexibility Integration

Reliability, redundancy upgradeability

Infrastructure Application Platform Data Platform

Delivering Cloud

Infrastructure Services Delivering Cloud Application Platform

Services

Delivering Big Data Platform Services

New Analyt

ics

Model

(15)

The physics of the Flex Chassis

(16)

2 nodes each with 2 CPU sockets, 24 DIMMs and 2 Hot- Swap Hard Drives 2 node Fillers

2 node Fillers

2 nodes without Hot- swap HDDs

Full Width node

Full Width node with up to 4 processor sockets, 48 DIMMs and Hot- Swap HDDs

2 CPU node with PCIe adaptor expansion node

Bay 1

Bay 14

Chassis Front View and Compute Node examples

(17)

Power Supplies (6X) IO Modules (4X) 40 mm Fans (2X)

CMM (2X) 80 mm Fan

Packs (8X)

Fan Mux Card (2X)

Chassis Rear View

4 2

3 1

1 6

1

10

(18)

Node Bay Shelf

Remove for Full Wide node Remove Two Shelves for Full Wide, Double High node

Chassis Shuttle

Chassis Bay Shelf Removal

(19)

Flex System Power – Overview

Flex chassis is optimized for 3-phase, 60A, 200-208VAC (US/Japan)

– Efficient use of 3-phase, 32A 220-240VAC / phase (International) Chassis contains up to six 2505W power supplies

– Supports both N+N and N+1 power redundancy

– Efficient balancing of N+N power across all 3 phases

– Power supply unit (PSU) is sized to efficiently use PDU power High Efficiency

– PSU certified to 80 PLUS Platinum

– Low power loss mid-plane and connectors

(20)

Cooling Paths – Zoned Cooling Between Subsystems

Switch Cooling

Node Cooling Power Supply Cooling

Zoned Cooling:

• Nodes

• Left / Right

• •Subsystems

• Node, Switch, and PSU Modules

(21)

The networking in the Chassis

(22)

I/O Adapter

1 I/O

Adapter 2

Node I/O: LAN-on-Motherboard (LoM) and I/O Adapter slots

x86 nodes have models with LoM and models without LoM, Power nodes only have models without LoM

(23)

CMM

L2 Switch

Flex System Manager Bay 1

IMM2

Eth LOM

L2 Switch

Eth

Compute Node x86 Bay 2

IMM2

CMM

L2 Switch Adapter 1 Adapter 2

Compute Node POWER Bay 14

FSP Adapter 1 Adapter 2

Switch Bay 1

Switch Bay 2

Switch Bay 3

Switch Bay 4

Management Network Data Network

I/O Bandwidth: “Future-proofed” mid-plane capability

(24)

CMM

L2 Switch

Flex System Manager Bay 1

IMM2

Eth LOM

L2 Switch

Eth

Compute Node x86 Bay 2

IMM2

Management Network Data Network

CMM

L2 Switch Adapter 1 Adapter 2

Compute Node POWER Bay 14

FSP Adapter1 Adapter 2

Switch Bay 1

Switch Bay 2

Switch Bay 3

Switch Bay 4

In summary … a very capable multi-layered mid-plane design

(25)

S y s te m i n fr a s tr u c tu re

Flexible networking solution, allowing for best price/performance

IBM 10Gb Switch: Wired for up to 16 10Gb ports per node and twenty two external ports

Networking

NIC

Mezz card-1

Mezz card-2

Midplane CPU-Node

Up to 4 KR Ports Per Switch Bay

NIC

Scalable Switches 5,6

Scalable Switches 7,8 Scalable Switches 3,4 Scalable Switches 1,2

4-port Mezz

Uplink Ports

ScS – Bay 1

ScS – Bay 2

(A) (B)

2 going to 4 per physical bay

8 going to 16 ScS – Bay 3

ScS – Bay 4

NIC

(26)

Ethernet & FCoE Fibre Channel InfiniBand – 52 port 1Gb Switch

• Base: 14/10 (internal/external)

• Upgrade: 14/10

• Upgrade: four 10Gb uplinks

– 64 port 10Gb Ethernet Switch

• Base: 14/10

• Upgrade: 14/8 (two 40Gb uplink)

• Upgrade: 14/4 – 1/10Gb Pass-Thru

– 20 port 8Gb (Qlogic) – 20 port 8Gb Pass-Thru

(Qlogic)

– 12 port 16Gb (Brocade) – 24 port 16Gb w/ ESB

(Brocade)

QDR Switch (Mellanox) – Upgrade: FDR

– 4 port 1Gb (Broadcom) – 4 port 10Gb (Emulex) – 2 port 10Gb (Mellanox)

– 2 port 8Gb (Qlogic) – 2 port 8Gb (Emulex) – 2 port 16Gb (Brocade)

QDR & FDR Adapter (Mellanox)

S w it c h A d a p te r

FlexChassis I/O Portfolio view

(27)

IBM 10Gb Scalable Switch for IBM Flex System Chassis

One, two, or three 10G ports per server – selectable by software license

Base Switch: 14 x 10Gb server port and 10 x 10Gb uplinks Switch upgrade 1: 28 x 10Gb server ports and 16 x 10Gb uplinks. 40Gb uplinks

enabled.

Switch upgrade 2: 42 x 10Gb server ports and 22 x 10Gb uplinks. 40Gb uplinks

enabled.

1.28 Tbps – first 1Tbps+

blade switch

5+ Tbps per chassis

IBM Flex System 10Gb Virtual Fabric Scalable Switch

Two 40 G uplink ports. Each port **can also be converted to 4*10G**

using QSFP to SFP+ cable

Full featured, Scalable bandwidth bringing convergence and simplicity to Datacenter

applications

(28)

The Compute power in the

Chassis

(29)

S y s te m i n fr a s tr u c tu re

Compute

Diverse offerings to match the diverse workloads

System Portfolio tuned to workloads

◊

Reduce acquisition costs through virtualization

consolidation

◊

Maximum platform capability provides deployment

flexibility

IBM Flex System x220

IBM Flex System p260

IBM Flex System p24L IBM Flex System x240

IBM Flex System p460 IBM Flex System

x440 Compute Node

IBM Flex System PEN

(PCIe Expansion Node)

(30)

• Standard Compute Node Form Factor

• 2-Socket Sandy Bridge-EP (Xeon E5-2600 Series Processor) 135W, 130W, 115W, 95W, 80W, 70W, 60W (4 – 8 core)

• 24 LP DDR3 DIMMs / Up to 1600MHz

• Up to 384GB Memory capacity with 16GB RDIMMs

• Up to 768GB Memory capacity with 32GB LR-DIMMs

• Chipset – Patsburg B (Intel C600 Series)

• 2x2.5” Hotswap SAS/SATA/HDD/SSD

• Onboard ServeRAID H1135 SAS Controller (RAID 0, 1) 6Gbs + Optional ServeRAID M5115 (RAID 0,1,5,6,10,50) with Flash-to- cache, supports eight 1.8inch SSDs

• SAS or SATA drives

• LSI 2004 connects to Patsburg with x4 PCIe Gen2

• Emulex BE3 10Gb Dual Ethernet LOM using Periscope connector

• LOM supports iSCSI and FCoE with FoD (feature On Demand) option

• Models with LOM and LOM-less

• Up to 2 Fabric Mezzanine Cards (PCIe Gen3 – 8GTs capable)

• Each Mezz connector: 1x16 PCIe port and 1x8 PCIe port

• Installing Mezz 1 card requires removal of LOM Periscope connector

• uEFI / IMMv2 / TPM 1.2 Rev 1.03

• Power Management

• xSmartEnergy Control

• Capping w/Pstate and SCI, Sys Power Maximizer

• Embedded Hypervisor

• ESXi on Flash key option (2 USB Keys for redundant boot option supported on unique DIMM air baffle)

• Front panel – one USB connector and one dongle

• Dongle includes two USB, video, and one serial port

• Front panel connector does not have enough power to run external USB drives

• Management

• iMM V2 Management Controller

• RTMM for Power Exec and Power Sequencing

x240 Compute Node Overview

(31)

PCIe x1 LPC

USB

iMM v2

QPI Links

DDR3 DIMMS

Intel SandyBridge

EP Processor 2

DDR3 Bus G DDR3 Bus H DDR3 Bus E DDR3 Bus F

DDR3 DIMMS DDR3 DIMMS DDR3 DIMMS DDR3 DIMMS

DDR3 Bus C DDR3 Bus D DDR3 Bus A DDR3 Bus B DDR3 DIMMS

DDR3 DIMMS DDR3 DIMMS

NC (8 PCIe lanes + unused x4 ESI) x4 ESI

Cntl, Misc

HS HDD/SSDs

Patsburg SSB

(C600 Series)

LSI 2004 SAS

Management to Midplane

x16 PCIe Gen3 x16 PCIe Gen3

Intel SandyBridge

EP Processor 1

FRM or PME/SME Interface

ETH Out ETH Out PCIe x2

Internal USB

USB USB(2)

Dual KR to Midplane x4 PCIe

Gen2

x8 PCIe Gen3

Mezz 0

10Gb LOM

ETH In ETH In

Video (RGB)

Front USB

Front DongleKVM

USB(2)

Mezz1 Mezz2

PHY PHY Serial Port

USB

QPI 1.1 – 8GT/s DDR3 Ch – up to 1600MT/s

PCIe Gen3 – 8GT/s

SPI

TPM1.2

System BIOS 16MB

SPI Parallel

Exp.

NAND Flash 2GB pbDSA iBMC Flash

8MbitBoot

RTMM Power Seq.

Power Exec

DDR3 Video 256MB

I2C

I2C LPC LPC

TPM IMM

x8 PCIe Gen2

QPI – 8.0GT/s

16GB/s/dir

16GB/s/dir 8GB/s/dir

4GB/s/dir

DDR3 – 1600MT/s/Ch DDR3 – 1600MT/s/Ch

Pop/No-Pop for FRM

X240 Compute Node Block Diagram

(32)

Flex x86 Node Relative Performance

(33)

S y s te m i n fr a s tr u c tu re

Storage Node Design Overview

Storage

In Chassis Enterprise Storage Double high / double wide node Dual hot swappable controllers for HA Shared storage for multiple CPU nodes High performance/ high function storage

Midrange class performance FCoE & iSCSI block storage

FC block storage as well NAS file storage (future)

24 HDDs/SSDs – 2.5” hot swap disk trays Via dual SAS paths to both controllers Add’l storage via Storage Disk Expansion Automated FW for hot spot data migration

Migrate content between SSDs and HDDs

Chassis and Management Appliance Integration Tight integration with chassis mgmt

Tight integration for storage and fabric mgmt VM provisioning, copy svcs, mirroring, etc

(34)

34

Enclosure Internal layout

Full Wide

24, 2.5” HDD Bays Controller

Full Wide

Double High

24, 2.5” HDD Bays Controller

Node Canister

Enclosure

Hard Disk Bay

HIC card

(35)

Management of the hardware

(36)

(37)

S y s te m i n fr a s tr u c tu re

Management

Simplified Virtualization Management experience

P h y s ic a l V ir tu a l W o rk lo a d

Physical Consolidation

& Setup Management Integration Resource Utilization Resource Pooling Intelligent Automation

Choice of Hypervisors

Automated virtual machine placement Dynamic allocation of virtual server, storage and network resources

Network QoS

Virtual workload definition

Placement services and advisors

Pooling of all network switch resources to enable consistent network policy application

Monitors VM network utilization within the fabric Virtual Machine relocation for compute, storage and networking

Monitors the Network Fabric for congestion

(38)

FSM Flex System Explorer

Dashbar Explorer Hardware View

Chassis only, otherwise Explorer shows a table

Rich Menus

Finder Status Pods

Framework Page

(39)

The Cloud software stack

(40)

IBM SmartCloud Entry solution architecture, concept

Power Systems Reference Configurations

IBM Systems Director 6.2

PowerVM ESXi

VMControl

2.3.1 VMware

vCenter VMControl API vCenter API

Cloud administration and management

Tivoli Provisioning

Manager for Images 7.1.1

REST API Self-service UI

IBM SmartCloud Entry management stack

Customer integration

• Approval policy, project management, users and roles

• Events and auditing

• Metering

• Image library

Common functionality Power Systems only System x only

System x

Reference Configurations

(41)

IBM SmartCloud Entry solution architecture, concept

Power Systems Reference Configurations

PowerVM ESXi

VMControl

2.3.1 VMware

Tivoli Provisioning

IBM SmartCloud Entry management stack

• Metering

• Image library

System x

Reference Configurations

(42)

(43)

Network sample

(44)

IBM SmartCloud Entry solution architecture, concept

Power Systems Reference Configurations

PowerVM ESXi

VMControl

2.3.1 VMware

Tivoli Provisioning

IBM SmartCloud Entry management stack

• Metering

• Image library

System x

Reference Configurations

(45)

vCenter API

The Vix API is object-based. Most API functions either create objects or operate on the properties of existing objects.

Client applications reference Vix objects with handles. Handles are opaque identifiers (actually integers) that can be passed as parameters to functions. Handles are run-time only and are unique only within a client's address space.

Most functions in the C-language API take a handle as a parameter. Because a handle value represents an object to the API, this document uses the terms "handle" and "object"

interchangeably.

There are several handle types, but a few of the key types are:

– Virtual Machine -- A single virtual machine, which might or might not be powered on.

– Host -- A single host computer, either the local host or a remote host.

– Job -- An object used in managing asynchronous operations.

– Snapshot -- A snapshot of a virtual machine.

(46)

IBM SmartCloud Entry solution architecture,concept

Power Systems Reference Configurations

PowerVM ESXi

VMControl

2.3.1 VMware

Tivoli Provisioning

IBM SmartCloud Entry management stack

• Metering

• Image library

System x

Reference Configurations

(47)

REST API reference

IBM SmartCloud Entry provides the following services for appliance libraries.

GET /appliances: Retrieve appliances from the cloud.

GET /appliances/{id}: Retrieve specific appliance from the cloud.

PUT /appliances/{id}: Update specific appliance in the cloud.

DELETE /appliances/{id}: Delete a captured appliance that has failed.

GET /appliances/{id}/targets: Retrieve the targets that can handle a workload of this appliance.

GET /appliances/{id}/customization: Retrieve default customization available when deploying this

appliance.

PUT /appliances/{id}/customization: Update the default customization for this appliance (admin

service).

DELETE /appliances/{id}/customization: Resets the customization for this appliance.

GET /appliances/{id}/log: Retrieve a appliance's capture progress logs.

POST /appliances: Create a new appliance by taking a workload capture.

The representational state transfer (REST) application programming interface (API) is

provided by SmartCloud Entry.

(48)

IBM SmartCloud Entry solution architecture, concept

Power Systems Reference Configurations

PowerVM ESXi

VMControl

2.3.1 VMware

Tivoli Provisioning

IBM SmartCloud Entry management stack

• Metering

• Image library

System x

Reference Configurations

(49)

49

IBM SmartCloud Entry: Comprehensive cloud capabilities

Administrator Functions

• Configure workload to host

• Configure workload targets to system pool

• Configure Virtual Appliance Parameters

• Register Virtual Appliance for User Selection

• Approve/Reject New Workload Requests

• Approve/Reject Workload Resize Requests

• Configure Billing

• Charging accounts, account assignment

• Configure approvals

• Configure to generate metering records

• Create & Manage Projects

• Add users to projects as admin, deployer, viewer

• Create Users

• Configure Network Pools

• Configure LDAP environment

• Cloud Configuration to VMControl or VMware

• Manual Intervention

• Review event logs & failures

• Initiate workload movement (between projects or via virtualization manager)

End User Functions

• Request access to a project

• Request CFS access from login panel

• Request workload creation – appliance deploy

• Set user instance parameters (CPU, memory)

• Resize Running Workloads

• Delete a Workload

• Clone a workload

• Start/Stop a workload

• Review workload properties

• Add additional disk (VMControl)

• Set target disk size at deploy

(VMware)

(50)

IBM SmartCloud Entry delivers cloud experience to users

Easy to access, easy to use Service Request Catalog

Hides underlying infrastructure from user and shifts focus to services delivered Enables the ability to provide standardized and lower cost services

Facilitates a granular level of services metering and billing

Workload standardization eases complexity

(51)

The IBM SmartCloud Entry end-user self-service scenario

End

Users Service Portal

Service Request Catalog

Provisioning Engine Workflows

Expert Systems Scripts

Service Modules Metering/Usage Billing

Approvals

Virtualized Cloud Infrastructure

Easy to access, easy to use Service Request Catalog Hides underlying complex infrastructure from user and shifts focus to services provided

Enables the ability to provide standardized and lower cost services

Facilitates granular level of services metering and billing

Workload standardization eases complexity

(52)

Self-service Web UI Basic usage metering

IBM SmartCloud Entry delivers cloud benefits to managers

(53)

Wrap up

(54)

IBM SmartCloud Entry solution architecture, concept

Power Systems Reference Configurations

PowerVM ESXi

VMControl

2.3.1 VMware

Tivoli Provisioning

IBM SmartCloud Entry management stack

• Metering

• Image library

System x

Reference Configurations

(55)

ISV and IBM Patterns

SmartCloud Provisioning

Compute Network

Storage

SmartCloud Provisioning

SmartCloud Entry

Compute Network

Storage

SmartCloud Provisioning

SmartCloud Entry

Pre-NGP or Non IBM HW

Flex System PureFlex PureApplication

Compute Network

Storage

Compute Network

Storage

Pre-integrated entry point Upgrade opportunity

Platform and Infrastructure Management SmartCloud

Orchestration

SmartCloud Orchestration ISV and IBM

Patterns

ISV and IBM Patterns

ISV and IBM Patterns ISDM or

SmartCloud Orchestration

Orchestrates

Foundation

Patterns

(56)

Links

IBM SmartCloud Entry Documentation Representational state transfer - REST

Redbook - Implementing SmartCloud Entry on IBM PureFlex System

Redbook - ReadyPack for Cloud with Hyper-V on IBM Flex System

(57)

QUESTIONS ?

(58)

Ib F. Kristensen

IT Architect

Nordic PureFlex Product Manager Systems & Technology Group

Bytoften 1 8240 Risskov +45 2880 6217 ibfk@dk.ibm.com