Configuration and Administration Guide


Table of Contents

Audience

This document provides design, implementation and deployment instructions for the Microsoft CityNext Big Data Solution Accelerator. The audience for this document include application specialists, IT administrators and operators who are ready to deploy the solution accelerator.

Design

Design Goals and Considerations

Virtualization

2.1.1 Virtualization image.png
With virtualization technology, the Microsoft CityNext Big Data Solution Accelerator is designed with portability across private cloud, public cloud and hybrid cloud. This document focuses on the design, administration and deployment of the solution accelerator services and databases.

Security

2.1.2 Security image.png
2.1.2 Security image b.png
2.1.2 Security image c.png

System Security

  • Isolate physical and virtual networks
    • The solution accelerator reference architecture provides portability across on-premises, hybrid cloud and public cloud. The design isolates physical hosts and virtual machines; specifically to ensure services are physically independent. There is no network access between the physical and virtual network. Both the physical and virtual layers have their own infrastructure, monitoring, authentication and authorization mechanisms.
  • Disable unused features
    • Unused OS roles, features, services and ports are disabled in each virtual machine. Additionally, all servers and virtual machines are required to use the Windows Server 2012 Datacenter as the operating system.
  • Minimize attacked surface
    • Based on the security design above, all inactive network entry points are shutdown or set as “deny any” in the firewall and ACL.
  • Minimum privilege
    • Except for the domain admin and system admin, dedicated user accounts are created for specific usage. The server account only owns the necessary power to handle the services, but has no more additional privileges.
  • Specific zone policy
    • Each virtual machine has a specific security policy determined by which zone it drops in.

Network Security

  • VLAN isolation for each zone (FE, APP, DB and Infra) with bidirectional access. ACL is configured to allow limited access between specific zones.
  • VLAN isolation for Infra zone with bidirectional access to FE, APP, DB and Infra zone with limited TCP ports only.
  • VLAN access log.

Transmission Security

  • HTTP w/ SSL – 265 bit AES
  • TDS w/ SSL – 128 bit
  • Network Service Account
  • PKI Encrypt and Signature

Access Security

  • For public client
    • Public client authenticates via external authentication providers.
    • Public client access with external authentication tokens.
    • Service broker validates the external authentication tokens and maps it with a domain account.
    • Service Broker routes the request to an App Service via impersonation to Entity Access.
    • Entity Access impersonates the client’s credentials to access data via Views.
    • Access log.
  • For domain client
    • Service broker routes the request to an App Service via impersonation to Entity Access.
    • Entity Access impersonates the client’s credentials to access data via Views
    • Access log.
  • Minimum privilege
    • Dedicated service user accounts are created for specific usage. The service account only owns the necessary power to handle the service, but has no more additional power.

Data Security

  • Data at Rest Encryption
    • Network packages are encrypted by HTTPS.
      • HTTP w/ SSL – 265 bit AES
      • TDS w/ SSL – 128 bit
      • PKI Encrypt and Signature
    • Configuration database which stores environment related configurations is encrypted – 128 bit AES.
  • Backup and Restore
    • CityNext data (SQL data, HBase data and Hive data) backup daily and restore on demand.

Availability

Service Availability

  • Web services are replicated by Windows Network Load Balancing to provide active-active availability.
  • The Service Bus is replicated by service bus farm management with active-active availability.
  • The SQL Server’s ingestion services and agent services are replicated via the pub/sub mechanism with active-active availability.
  • The SQL Server’s linked server, report server and analytic server are replicated by SQL multiple instances with active-active availability.
  • The SharePoint Server is replicated by the SharePoint cluster.
  • The Active Directory is replicated by the Active Directory cluster.
  • The DNS service is replicated by the DNS cluster.
  • The SCOM is replicated by the SCOM cluster.
  • The Analytic nodes rely on the virtualization layer to provide active-passive availability.
  • The Power BI Gateway relies on the virtualization layer to provide active-passive availability.

Data Availability

  • SQL data is replicated by the SQL AlwaysOn Group. SQL instance failure will trigger a replicated SQL instance to be activated as the primary read/write instance.
  • HBase and Hive data are replicated by Hadoop Distributed File System. Any data storage failure will trigger a live move in order to rest available ones for data blocks.

Scalability

Service Request Scalability

  • Web services are scaled by Windows Network Load Balancing.
  • The Service Bus is scaled by service bus farm management.
  • The SQL Server’s ingestion services and agent services are scaled via the pub/sub mechanism.
  • The SQL Server’s linked server, report server and analytic server are scaled by multiple SQL instances.
  • The SharePoint Server is scaled by the SharePoint Cluster.
  • Analytic nodes are scaled via the pub/sub mechanism.
  • The Power BI Gateway does not currently support scale out.

Data Volume Scalability

  • Hive/HBase data volume can be extended until it reaches HDFS maximum capacity (100+ Petabytes).
  • SQL data volume can be extended until it reaches SQL maximum capacity (16 TB per data file, 2 TB per log file).

Back to top

Zones

2.2 Zones image.png
2.2 Zones image b.png
2.2 Zones image c.png

Front-End Zone

Goal

  • Access Internet and publish CityNext Services
    • Service Broker
    • Data Ingestion Pull Channels
    • Data Ingestion Push Channels
    • Information Dissemination
    • Power BI Gateway
  • Isolated with Data Zone

Zone Capability

2.2.1.2 Zone Capability.png

Zone Security Rules

2.2.1.3 Zone Security Rules.png

Application Zone

Goal

  • Hosts multiple services which are not required to access or be accessed by the Internet, but are required to access the CityNext Data Store.
    • Internal services
    • Management services
    • Analytics nodes
    • Agent services
    • SQL Server’s linked server, reporting service, and analytic service
    • SharePoint Server
    • Service Bus

Zone Capability

2.2.2.2 Zone Capability.png

Zone Security Rules

2.2.2.3 Zone Security Rules.png

Data Zone

Goal

  • Data Storage
    • Business data (structured and unstructured)
    • Management data
    • Telemetry data
  • Provides data access to the application zone

Zone Capability

2.2.3.2 Zone Capability.png

Zone Security Rules

2.2.3.3 Zone Security Rules.png

Infrastructure Zone

Goal

  • Infrastructure support for all zones
    • Active Directory
    • DNS
    • Windows Server Update Service
  • CityNext Monitoring Service
    • Monitoring Service
    • System Center Operation Manager
  • CityNext Operation Entry
    • Management Studio
    • Management PowerShell Console

Zone Capability

2.2.4.2 Zone Capability.png

Zone Security Rules

2.2.4.3 Zone Security Rules.png
Back to top

Software Requirements

2.3 Software Requirements.png

Virtual Machine Capacity

Capacity Profiles

2.4.1 Capacity Profiles.png

Private Preview VM Capacity

2.4.2 Private Preview VM Capacity.png

Groups & User Accounts

2.5 Groups & User Accounts.png

IP Allocation

2.6 IP Allocation.png

Deployment Preparation

Configure Azure Mobile Service

  • Create a new Mobile Service through https://manage.windowsazure.com.
  • Create tables “App”, “ChannelUri”, and “Log” through the DATA tab.
  • Create APIs manually through the API tab. The APIs are located at {ReleaseFolder}\V1PrivatePreview_Src\Source\Core\InfoDissemination\Channels\ToastNotification\WAMS\service\api
    • For each .js file, create an API with same file name in Mobile service (Example, for app.js, API name is 'app'. Permission should refer to app.json). Copy the js content and replace the default script content of that API.
  • Update Configuration DB
    • Update value of “WAMS.ApplicationKey” to the mobile service key
    • Update value of “WAMS.ToastServiceUrl” to the mobile service URL
  • For any app that needs to receive a message from the solution accelerator
  • Set CLIENT ID and CLIENT SECRET through the IDENTITY tab and PUSH tab.
  • To get CLIENT ID and CLIENT SECRET, refer to http://msdn.microsoft.com/en-us/library/cc287659.aspx

AD and DNS Preparation

Basic

  • Create a small VM with Windows Server 2012 or Windows Server 2012 R2, enable AD and DNS feature.
  • If there is no available domain, create a domain called “citynext.com”, otherwise use current domain.
  • Create domain user “<domainname>\cigdev” with password “passw0rd!”.
  • Add cigdev as a domain admin.
  • Create domain group “<domainname>\analyticsusers”.

Configure Authorization Users and Groups

  • Create domain user “<domainname>\core” with password “passw0rd!”.
  • Create domain user “<domainname>\cityanonymous” with password “passw0rd!”.
  • Create domain user “<domainname>\normal” with password “passw0rd!”.
  • Create domain user “<domainname>\citizen” with password “passw0rd!”.
  • Create domain group “CitizenUsers_All” with members “cityanonymous, normal”.
  • Create domain group “CityUsersAll” with members “CitizenUsersAll”.

Configure for DNS

  • Add "ConfigurationServiceHost" with IP “127.0.0.1” in DNS.

Set SPN for Kerberos Authentication

  • Log into the Active Directory machine with the domain administrator
  • Open administrator PowerShell, run
    • setspn -S HTTP/<CityNexthostname> <domainname>\cigdev
  • Open "active directory user and computers"
    • <domainname> -> users -> cigdev
    • Right click cigdev, go to "delegation" tab
    • Select "trust this user for delegation for specific service only" -> "use any authentication protocol"
    • Click add, type in the VM host name
    • Choose "http" 'mssql" "mssql 1433" and add
    • Click apply, click ok

CityNext VM Preparation

Basic

  • Provision VM with template
  • Add VM to your domain
  • Log in with <domainname>\cigdev account, ensure this account is local admin/domain admin
  • Copy latest deployment script from \\{ReleaseFolder}\ V1PrivatePreview\Platform\Deploy\ to c:\deploy\scripts
  • Open the PowerShell command window with administrator privilege, run:
    • C:\deploy\scripts\deploy.ps1 –source c:\deploy\builds
    • C:\deploy\scripts\deploy.ps1 sqladminuser
  • Keep in mind that all ps1’s should run under administrator privileges

Configure Windows Authentication Prerequisites

  • Assign <domainname>\cigdev account “Act as part of the operating system” user rights
  • From “Start -> Run …”, type “gpedit.msc”
  • Select “Local Computer Policy” -> “Computer Configuration” -> “Windows Settings” -> “Security Settings” -> Local Policies” -> “User Rights Assignment” in the tree
  • Double click “Act as part of the operating system”
  • Click “Add User or Group …”
  • Type name “<domainname>\cigdev” and click “OK”
  • Restart the machine

Install HDP

  • Open PowerShell command window with administrator privileges
  • Run “C:\deploy\scripts\DeployHDP.ps1”

SharePoint – Basic

  • Open SharePoint Configuration Wizard
    • Page 1
      • Create a new server farm
    • Page 2
      • Database server: <CityNexthostname>
      • Username: <domainname>\cigdev
      • Password: passw0rd!
    • Page 3
      • Passphrase: passw0rd!
    • Page 4
      • Port: 46081
  • Go to SharePoint central administration
    • Create a new web application, choose default setting, click ok
    • Create a new site collection
      • Title = CityNext
      • Administrator: <domainname>\cigdev
      • Click ok

SharePoint – CA

  • Copy AnalyticsSPSchema.wsp from C:\deploy\builds\DailyCityAnalytics\1.0.0403.1\Release to SharePoint C:\deploy\builds\DailyCityAnalytics\1.0.0403.1\Release\AnalyticsSPDeploymentPS
    • Ensure that both deploy.ps1 and WSP files have the same directory path
  • Open SharePoint PowerShell as administrator, run
    • cd C:\deploy\builds\Daily_CityAnalytics\1.0.0403.1\Release\AnalyticsSPDeploymentPS\
    • deploy.ps1 -web http://<CityNexthostname>/
  • Open http://<CityNexthostname>/
  • Go to gear icon -> site settings
    • Go to "people and groups", select “groups”, click “new”
      • Create group "AnalyticsUsers"
      • Add users who want to access SharePoint or create a job
    • Go to "Site Permissions"
      • Click grant permission
      • Grant "AnalyticsUsers" permission of "Contribute"
  • Make sure the SharePoint - 80 site Windows Authentication is enabled and using “Negotiate” in providers
    • Select the SharePoint – 80 site
    • Right click Windows Authentication => Providers
    • Select “Negotiate” as available provider, and click the “Add” button
    • Select "Enable Kernel-mode authentication"
3.3.5 SharePoint – CA image .png

SharePoint - CSM

  • Copy CoreDataModel.wsp from C:\deploy\builds\DailyCore\1.0.0403.1\Release to C:\deploy\builds\DailyCore\1.0.0403.1\Release\CSMSharePointDeployPS
  • Open the SharePoint 2013 Management Shell as Administrator
  • Deploy core data model, enter the command below:
    • C:\deploy\builds\Daily_Core\1.0.0403.1\Release\CSMSharePointDeployPS\DeployCSMCoreDataModel.ps1
  • Initialize core data model, enter the command below:
    • C:\deploy\builds\Daily_Core\1.0.0403.1\Release\CSMSharePointDeployPS \InitCSMCoreDataModel.ps1 –web http://<CityNexthostname>/ –feature coredatamodel
  • Run the ImportCategoryAndServiceType.ps1:
    • C:\deploy\tools\csm\ImportCategoryAndServiceType.ps1 -SharePointSiteUrl http:// <CityNexthostname>/ -CategoryPath C:\deploy\tools\csm\Category.cmp -ServiceTypePath C:\deploy\tools\csm\Servicetype.cmp
  • Enable anonymous access to SharePoint Image Library
Step 1:
Open SharePoint Central Admin Application Management ->Manage web applications ->Select the web application ->Click the Authentication Providers ->then choose “Enable anonymous access”
3.3.6 SharePoint - CSM image.png

Step 2:
Open IE, go to http://<citynexthostname>Site Permission -> anonymous access -> Lists and libraries
3.3.6 SharePoint - CSM image b.png

Step 3:
Click left panel, recent -> citynextimages
Select “library” tab, click library settings
Click “permissions for this document library”
Click “stop inheriting permissions”
3.3.6 SharePoint - CSM image c.png
3.3.6 SharePoint - CSM image d.png

Configure SQL Server Reporting Service

  • Configure SSRS:
    • Open "Reporting Service Configuration Manager"
    • Connect
    • Select "Service Account" -> use another account -> "<domainname>\cigdev", click apply
    • Select "Web Service Url", click apply
    • Select "Database" -> Create new database -> click next to the end
    • Select "Report Manager Url", click apply
    • Visit http://localhost/Reports in an IE browser and click the "New Folder" button in the top left navigation bar and the name is "citynext" (if any UAC errors occur, try to go to another server in the same domain)
  • Verification:
    • http://localhost/Reports, a UAC error or normal page is expected

Configure Linked Server

  • Go to the local SQL, create a new DB "PowerBI"
  • Ensure Hive ODBC is there
  • Run the scripts below to create linked server:
EXEC master.dbo.sp_addlinkedserver
@server = N'HiveDW', @srvproduct=N'HIVE',
@provider=N'MSDASQL', @datasrc=N'<hive odbc DSN name>',
@provstr=N'Provider=MSDASQL.1;Persist Security Info=True;User ID=sa;Password=passw0rd!'

Configure Service Bus

  • Run Service Bus Configuration from the start menu
  • Create a new farm, use default configuration
  • SQL server instance:
    • Use Windows Authentication
    • Certificate generation key: 78d987wA+D&AWd7=9sad7=8da=w87dsadyh28u123
    • Enable firewall rules on this computer: CHECK
    • Create a default namespace: CHECK
    • Manage this farm through the dev portal: CHECK
      • Admin: <DOMAINNAME>\cigdev
      • Tenant: create a new local account called ServiceBusTenant (password: passw0rd!)
  • After Service Bus installation:
    • Generate certificate under ServiceBus Admin account from <ServiceBusHost> by Get-SBAutoGeneratedCA
      • Root public certificate file: AutoGeneratedCA.cer
      • Root certificate revocation list file: AutoGeneratedCA.crl
  • Import this certificate into local under Trusted Root Certification Authorities with Machine level

Install SQL Server CE for StreamInsight

  • Install the following two executable files in sequence:
    • C:\Program Files\Microsoft StreamInsight 2.1\Redist\SSCERuntime_x86-ENU.exe
    • C:\Program Files\Microsoft StreamInsight 2.1\Redist\SSCERuntime_x64-ENU.exe

Back to top

Deployment

Run Deployment Script

  • Log into the CityNext VM with <domainname>\cigdev
  • Open the PowerShell command line with administrator privilege
  • Run “C:\deploy\scripts\DeployOneboxV1PrivatePreview.ps1”

Configure Analytic Service

  • For C:\AnalyticsResultStorage\
    • NTFS security (Properties -> Security -> Advanced)
    • Disable inheritance (by converting, not removing inherited permissions)
    • Remove all permissions except SYSTEM and Administrators
    • Add <domainname>\cigdev with Full Control
  • Double click C:\CityNextBDP\bin\PsExec.exe once. A EULA dialog box will appear, click agree

Configure Monitoring Service

  • Update VM capacity value in C:\CityNextBDP\www\MonitoringService\App_Data\VMList.xml
    • MemoryCount (GB)
    • StorageSpaceCount (GB)
    • NetworkOfGatewayCount (MB)

Configure SSIS

  • Open SQL Server Management Studio
  • Right click “Integration Services Catalogs” -> “SSISDB”
  • Change “Retention Period” from 365 to 1
  • Click OK

Restart the Server

Back to top

Post Deployment Verification

  • Copy the post deployment verification tool from {ReleaseFolder}\Deploy\scripts\verification to c:\deploy\scripts\verification
  • Open the PowerShell prompt with administrator privilege. run “c:\deploy\scripts\verification\scripts\RunTest.ps1 -All”, it will take 15-30 minutes
  • Check result at report.trx.htm. If they all pass then deployment was successful
  • Debug error:
    • See details in result.trx
    • Rerun individual test/sequenced test by
      • RunTest.ps1 –Test “<test>”
      • RunTest.ps1 –OrderedTest “<orderedtest>”
        • Press “tab” after “-orderedtest” to switch across all available ordered tests

Back to top

Last edited Aug 1, 2014 at 7:58 AM by gheadd, version 10