Configuration and Administration Guide


Table of Contents

Audience

This document provides setup and deployment instructions for the Microsoft CityNext Big Data Solution Accelerator. The audience for this document includes IT administrators and operators who are ready to setup the infrastructure and deploy the Microsoft CityNext Big Data Solution Accelerator.

Prerequisites

Minimum Hardware Requirements

Standard size:
  • A2 = 2 cores, 3.5 GB
  • A3 = 4 cores, 7 GB
  • A4 = 8 cores, 14 GB
  • A5 = 2 cores, 14 GB

Minimum H/W (W/O secondary SCOM):
  • CPU: 66 Cores
  • Memory: 171.5 GB
  • HD: 9520 GB
    • OS Disk: 2520 GB
    • Data Disk: 7000 GB
2.1 Minimum Hardware Requirements.png

Minimum Software Requirements

2.2 Minimum Software Requirements.png
2.2 Minimum Software Requirements 2.png

Minimum Azure Quota Requirements

  • 68 CPU Cores
  • 22 Virtual Machines
  • 15 Cloud Services
  • 1 Storage Account (Local-Redundant or Geo-Redundant)
  • 1 Virtual Network

Topology and IP Allocation

2.4 Topology and IP Allocation.png


2.4 Topology & IP Allocation 2.png


Service Accounts and Groups

2.5 Service Accounts & Groups.png

User Accounts and Groups

2.6 User Accounts & Groups.png

Back to top


Setup Azure Infrastructure

Create Azure Subscription

Make sure that you have a valid Azure subscription with enough quota mentioned in Minimum Azure Quota Requirements. To create a new Azure subscription, click here https://account.windowsazure.com/Home/Index.

Create Affinity Group

  • Log in to manage.windowsazure.com with the administrator account
  • Settings -> Add
  • Input the Name “CityNextAffinity”, Description
  • Select the closest region to your clients, click OK

Create Azure Storage

  • Log in to manage.windowsazure.com with administrator account
  • Storage -> New -> Quick Create
  • Input the URL, such as “citynextstorage”; set the affinity group to the one defined in section "Create Affinity Group"
  • Select the replication option by local redundant or geo-redundant
  • Click create

Create Azure Virtual Network and DNS

  • Log in to manage.windowsazure.com with the administrator account
  • Virtual Networks - > New -> Custom Create
  • Input the Name, such as “CityNextNetwork”; select the same region as the affinity group
  • Input the DNS server name “INAD01”; IP set to 192.168.1.199
  • Input another DNS server name “INAD02”; IP set to 192.168.1.200
  • Set Address Spaces as shown below; click create
3.4 Create Azure Virtual Network & DNS.png
Back to top

Create Azure Virtual Machines

  • Log in to manage.windowsazure.com with administrator account
  • Virtual Machine -> New -> From Gallery
  • Microsoft -> Windows Server 2012 R2 Datacenter -> Next
  • Input a valid virtual machine name (refer to the “VMName” column in the table below; replace “XX” accordingly), and select the virtual machine size; set the administrator user name as “citynextadmin”, the password is “citynext0!”; click Next.
  • Select “Create a new cloud service”. Input the cloud service name and refer to the “cloud service” column in the table below and replace “XX” accordingly. Select the virtual network name (refer to section "Create Azure Virtual Network and DNS"); select the virtual network subnet following the table below; select the storage account (refer to section "Create Azure Storage"); select “Create an availability set” and use the availability set name according to the table below.

3.5 Create Azure Virtual Machines.png
3.5 Create Azure Virtual Machines 2.png
Note: The VMs in the “VM Name” cell will share 1 cloud service. When you create other VMs for the same cloud service, please select the existing cloud service name and availability set name and verify that they share the same one.

  • Attach the data disk for DBHDP01~05, DBSQL01~02, INSOCM01 and INOP01
    • Select a virtual machine; click “ATTACH” -> Attach empty disk, use the external disk size and amount according to the table below
    • You can choose to use no RAID or RAID 0 for a higher disk IO. Pick alternative ways from the two options below:
      • Configure data disk with no RAID
        • Log on to each machine, disk management -> Create volume F:\ -> select quick format
        • Confirm that all data disks are mounted as “F:\” drive
      • Configure data disk with RAID 0 (optional)
        • If you want a higher disk IO, choose RAID 0, refer to the “RAID0” column
        • Log on to each machine, disk management -> Create stripped volume F:\ include all attached disk. (please select the simple volume in inop01) -> quick format
        • Confirm that all data disks are mounted as “F:\” drive
3.5 Create Azure Virtual Machines 3.png
3.5 Create Azure Virtual Machines 4.png
Back to top

Set Static IP

Get-AzureVM -ServiceName $ServiceName | Stop-AzureVM -Force
Get-AzureVM -ServiceName $ServiceName -Name $Hostname | Set-AzureStaticVNetIP - IPAddress "$IPAddress" | Update-AzureVM
Get-AzureVM -ServiceName $ServiceName -Name $Hostname | Set-AzureStaticVNetIP -IPAddress "$IPAddress" | Update-AzureVM
Get-AzureVM -ServiceName $ServiceName | Start-AzureVM
3.6 Set Static IP.png

Set Internal NLB

  • Run Windows Azure PowerShell ISE as the administrator
    • Create the internal network load-balance and attach the endpoint to the internal NLB set, please refer to the sample below:
3.7 Set Internal NLB.png
  • For further detailed information, please refer to the table below:
3.7 Set Internal NLB 2.png
Back to top

Reset Computer Name

  • Follow the steps below for every machine:
    • Log on to the VM using the local admin user “.\citynextadmin”
    • Reset the computer name according to the table below:
3.8 Reset Computer Name.png
Back to top


Setup Domain and Accounts

Setup AD and DNS

  • Log on to INAD01
    • Set the static IP on this machine, please refer to the image below:
4.1 Setup AD and DNS.png
  • Select "add role and features" -> "Active Directory domain Services" and "DNS Server"; click next until completion
  • Wait for features to be installed completely
  • Click "Promote this server to a domain controller" in the Server Manager Notification list
  • Select "Add a new forest" in the "Active directory Domain Services Configuration Wizard" and type the domain name as "CITYNEXT.COM"
  • Type the DSRM password, and click "Next"
  • Finish the AD Configuration Wizard. Ignore the warning “a delegation for this DNS server cannot be found”.
  • Select Server Manager -> Dashboard -> Tools, Open “DNS”
  • Select DNS -> INAD01, right click and select properties
  • Open the INAD01 properties, select the forward tab, click edit
  • Add IP address “8.8.8.8” to the forwarding server list. Click OK until completion.
  • Log on to INAD02
    • Set the static IP on this machine, please refer to the image below:
4.1 Setup AD and DNS 2.png
  • Select "add role and features" -> "Active Directory domain Services" and "DNS Server"
  • Click "Promote this server to a domain controller" in the Server Manager Notification list
  • Select “Add a domain controller to an existing domain”; the domain name is “citynext.com”; use “citynext\citynextadmin” to perform this operation.
  • Input the DSRM password “citynext0!”
  • Select replicate from “INAD01.citynex .com”
  • Click next until completion
  • Finish all the steps; restart the machine; reset the static DNS IP as shown below:

4.1 Setup AD and DNS 3.png
Back to top

Setup Accounts and Groups

  • Log on to INAD01; select "Active Directory Users and Computers"
    • Security Group: PlatAdmUsrGrp (Join the domain admin group)
      • User: Platadmin01
      • User: Platadmin02
      • User: Platscomacct
    • Security Group: PlatEndUsrGrp
      • User: Platend01
      • User: Platend02
    • Security Group: PlatOpsUsrGrp
      • User: Platops01
      • User: Platops02
    • Security Group: PlatSvcAcctGrp
      • User: PlatSvcAcct (Deny user permission to log on to remote desktop)
    • User: InfrSvcAcct (Join the domain admin group)
  • Deny user “PlatSvcAcct” and “InfrSvcAcct” permission to log on to the remote desktop
    • Log on to INAD01; open “Active Directory Users and Computers”
    • Open the tree view: AD -> citynext.com -> Users ->
    • Select “InfrSvcAcct”, right-click -> Properties -> Remote Desktop Service Profile
    • Enable “Deny this user permissions to log on to Remote Desktop Session Host Server”
    • Select “PlatSvcAcct”, right-click -> Properties -> Remote Desktop Service Profile
    • Enable “Deny this user permissions to log on to Remote Desktop Session Host Server”

Disable Automatic Windows Updates in Group Policy (Optional)

Note: This section is to confirm that each VM will not be restarted randomly due to Windows Update. If you do not choose to configure this section, you should configure WSUS.
  • Log on to the AD server and open Group Policy Management
  • Open the tree view: Forest -> Domains -> citynext.com; right click the domain name and select "Create a GPO in this domain, and Link it here"
  • Type a new GPO name as "DisableAutoUpdates"; click "OK"
  • Find the new GPO in the "Group Policy Objects" folder; right click "edit"
  • Open the tree view in the new window: Computer Configuration -> Policies -> Administrative Templates -> Windows Components -> Windows Update
  • Set "Configure Automatic Updates" to "Disabled

Join Domain

In order to join the domain do the following on every machine (except inad01 and inad02):
  • Log on to the VM using the local admin user “.\citynextadmin”
  • Open a PowerShell prompt as the administrator. Execute the following command:
netdom join $env:COMPUTERNAME /Domain:citynext.com /UserD:citynext\platadmin01 /PasswordD:citynext0! /reboot:0

Setup Firewall

  • Import the .wfw file attached to each machine

4.5 Setup Firewall.png
4.5 Setup Firewall 2.png
4.5 Setup Firewall 3.png
  • If the VM IP configuration is changed from this document:
    • Log on to dbhdp01, 02, 03, 04, 05
    • Edit firewall rule “AllowHDPAllMachine”
    • Change remote IP range to the valid 5 IPs for dbhdp01,02,03,04,05
    • Do this for all 5 VMs

Setup Share Folder

  • Log on to inop01
    • Create the folders below and share:
4.6 Setup Share Folder.png

Create iSCSI Target Service

  • Log on to apmgr01 and 02
    • Click Server Manager -> Tools -> the iSCSI Initiator
    • Click “Yes” in the jump box to open the “iSCSI Initiator Properties” window
    • Select the Discovery tab, click Discover Portal, type the discover target name as "inop01.citynext.com”
    • Click OK to end
  • Log on to inop01
    • Install iSCSI Target Service and create 3 virtual external storages for the IIS Cluster
    • Install the iSCSI target service role
      • Add Roles and Features -> Server Roles -> File and Storage Services -> File and iSCSI Services -> select iSCSI target Server -> click next until completion
    • Navigate to Server Manager -> File and Storage Services -> iSCSI
4.7 Create iSCSI Target Service.png
  • Repeat below steps 3 times to create 3 iSCSI disks
    • Click “To create an iSCSI virtual disk, start the new iSCSI Virtual Disk Wizard” -> new iSCSI virtual disk…
  • Select the disk location in F:\ (for the virtual disk name please refer to the following table, each disk size is 5GB):
4.7 Create iSCSI Target Service 2.png
  • Create a new iSCSI target: iiscluster-inop01
  • In Access Servers page, click Add button to add initiator ID:
    • Select apmgr01 in “Select from the initiator cache on the target server” and click okay
  • In Access Servers page, click Add button to add initiator ID:
    • Select apmgr02 in “Select from the initiator cache on the target server” and click okay
  • Click Next until completion
Back to top

Retrieve each internal load-balance IP and create host names in DNS server

  • Run Windows Azure PowerShell ISE as the administrator
  • Run the PowerShell script below to get each IP address
4.8 Retrieve each internal load-balance IP and create host names in DNS server.png
  • Log on to INAD01; select the “DNS”
  • Open the tree view: DNS -> INAD01 -> Forward Lookup Zones -> citynxt.com
  • Right-click -> New Host
  • Input the name as shown in the “Input host name” column, for the IP address please refer to the results above. Executes 5 times.
4.8 Retrieve each internal load-balance IP and create host names in DNS server 2.png
Back to top



Setup Software

Get the Latest Windows Update

  • Log on to each VM
  • Get the latest Windows Update and install

Enable .Net 3.5, 4.5 and IIS Features Without WebDAV

  • Log on to each VM (except inad01 and inad0)
    • Navigate to Administrator Management, and click Add Roles and Features
    • Select all items with .Net 3.4 and 4.5 in Features wizard page
    • Select all items with IIS Role without “WebDAV” in Role Services wizard page
    • Click Next until the installation is complete

Enable IIS Permission

  • Log on to each VM (except inad01 and inad02)
  • Change authentication section in C:\Windows\System32\inetsrv\config\applicationHost.config to:
<sectionGroup name="authentication">
<section name="anonymousAuthentication" overrideModeDefault="Allow" />
<section name="basicAuthentication" overrideModeDefault="Allow" />
<section name="clientCertificateMappingAuthentication" overrideModeDefault="Allow" />
<section name="digestAuthentication" overrideModeDefault="Allow" />
<section name="iisClientCertificateMappingAuthentication" overrideModeDefault="Allow" />
<section name="windowsAuthentication" overrideModeDefault="Allow" />
</sectionGroup>

Install MVC4

Enable Performance Counter Permission

  • Log on to fedi01, fedi02, apsvc01, apsvc02, apmgr01, apmgr02
    • Open a command prompt as the administrator; type "lodctr /r" and press ENTER (this command may fail on the first attempt, but you can try more times if necessary).

Install PsExec

Install AppFabric

  • Download the AppFabric 1.1 installation file from: http://www.microsoft.com/en-us/download/confirmation.aspx?id=27115
  • Log on to apsvc01
    • Open Windows Update and keep it on “checking for updates” status, then do the next step
    • Install AppFabric 1.1 with caching service, cache client, and cache administration
    • Configure AppFabric 1.1
      • Select citynext\infrsvcacct as Caching Service account
      • Select XML as Caching Service configuration provider
      • Select \\INOP01\AppFabricConfig as the storage location for xml
      • Select new cluster
      • Click next until completion
    • Open \\inop01\appfabircconfig\clusterconfig.xml
      • Change size to 1024
    • Open the Caching Administration Windows PowerShell as the administrator user
      • Execute the command: start-cachecluster
    • Set AppFabric caching windows service to start automatically, and start the service
  • Log on to apsvc02
    • Open Windows Update and keep it on “checking for updates” status, then do next step
    • Install AppFabric 1.1 with caching service, cache client and cache administration
    • Configure AppFabric 1.1
      • Select citynext\infrsvcacct as Caching Service account
      • Select XML as Caching Service configuration provider
      • Select \\INOP01\AppFabricConfig as the storage location for xml
      • Select join cluster
      • Click next until completion
    • Set AppFabric caching windows service to start automatically, and start the service
  • Open \\inop01\appfabircconfig\clusterconfig.xml
    • Change size with second node to 1024
    • Change APSVC02 leadhost = "false"
  • Set AppFabric caching windows service to start automatically, and start the service
  • Log on to apsvc01
    • Open the Caching Administration Windows PowerShell window as the administrator, run the scripts below:
New-Cache -CacheName DataQueryResultCache
Grant-CacheAllowedClientAccount -Account citynext\platsvcacct
Get-CacheClusterHealth
the 2 nodes are online with the cache and should be displayed (approximately a 10 minute wait)
  • Note: If the AppFabric installation failed (MSI error 1603), please remove the extra " in the Environment Variables PSModulePath
    • Go to Control Panel -> System -> Advanced System Settings -> On the Advanced Tab Select Enviroment Variables -> Under System Variables there is a Variable named PSModulePath, select it -> Click Edit -> Remove trailing " (at the end)
Back to top

Install ARR

  • Log on to fearr01, fearr02
    • Make sure IIS is installed and has a default web site (If a support SSL is needed, add binding https/443)
    • Download Web Platform Installer from: http://www.microsoft.com/web/downloads/platform.aspx
    • Use Web Platform Installer to search and install "Application Request Router 3.0"
    • Open IIS Manager, Under "Server Farms”, Select "Create Server Farm"
      • Create farm: fedi, server fedi01, fedi02
      • Create another farm: APSVC, server APSVC01, APSVC02
    • Open IIS Manager -> Site Root -> URL Rewrite
      • Double click "ARRFEDILoadBalance" rule
      • Change highlighted section and click apply
      • Make sure "ARRFEDILoadBalance" rule is above "ARRAPSVCLoadBalance"
5.8 Install ARR.png
5.8 Install ARR 2.png

Install IIS Failover

  • Log on to apmgr01, apmgr02
  • Install the failover clustering feature
    • Add the Failover Clustering feature to the VM. In the Initial Configuration Tasks window, click Add features.
    • In the Add Features Wizard select Failover Clustering as shown below; click next
    • In Confirm Installation Selections click install
    • When the Failover Clustering feature installation is completed, click close
  • Log on to apmgr01
    • Create the WSFC Cluster
    • Open Failover Cluster Manager -> Action -> Create Cluster; create a single-node cluster on one of the nodes apmgr01, cluster name apmgrfc
    • Set static IP to apmgrfc
      • Failover Cluster Manager -> Cluster Core Resources -> IP Address -> Properties -> Static IP Address -> 192.168.1.74, Network -> 192.168.1.64/26
      • Bring the cluster name online apmgrfc
    • Add the other node to apmgr02
  • Configure the IIS shared configuration on all cluster nodes
    • Internet Information Services (IIS) Manager -> click the server name node
    • Double-click the Shared Configuration icon
    • On the Shared Configuration page, click Export Configuration in the Actions pane (the right pane) to export the configuration files from the local computer to another location
    • In the Export Configuration dialog box, type the path of the file share (\\inop01\APMGRIISCache) in the Physical path box
    • Click Connect As, and then type the user name and the password for the user account that has access to which the shared configuration is stored, and then click OK. This account will be used for access, such as CITYNEXT\INFRSVCACT.
    • In the Export Configuration dialog box, type a password that will be used to protect the encryption keys (Citynext0!), and then click OK
    • On the Shared Configuration page, click to select the Enable shared configuration check box
    • Type the physical path, the user account, and the password that you entered previously, and then click Apply in the Actions pane
    • In the Encryption Keys Password dialog box, type the encryption key password that you set earlier, and then click OK
    • In the Shared Configuration dialog box, click OK
    • Click OK and restart the IIS Management
5.9 Install IIS Failover.png
  • Log on to apmgr02
    • Navigate to Administrative Tools, and then click Internet Information Services (IIS) Manager
    • Click the server name node
    • Double-click the Shared Configuration icon
    • On the Shared Configuration page, click to select the Enable shared configuration check box
    • Type the physical path of the file share (\\inop01\APMGRIISCache) the user account, and the password that you entered previously, and then click Apply in the Actions pane
    • In the Encryption Keys Password dialog box, type the encryption key password that you set earlier, and then click OK
    • In the Shared Configuration dialog box, click OK
    • Click OK, and restart the IIS Management
  • Log on to apmgr01; connect to iSCSI server; create disk volume
    • Navigate to Administrative Tools, and then click iSCSI Initiator
    • Select the Discovery tab, click Discover Portal, type the discover target name as "inop01.citynext.com" or using its IP address to connect
    • Select the Targets tab; click Connect button; click OK to finish
    • Navigate to Administrative Tools, and click Computer Management, open Disk Management
    • Select 3 unallocated hard disks; initialize these disks as the MBR disk; format them; then create 3 volume : E, F, G
    • Navigate to Administrative Tools, and click Failover Cluster Manager
    • Connect to the cluster. If you are on one of the cluster nodes, the cluster will appear on the list automatically.
    • Under Storage -> Disks, click “Add Disk” to add these disks
  • Log on to apmgr02; connect to the iSCSI server
    • Navigate to Administrative Tools, and then click iSCSI Initiator
    • Select the Discovery tab, click Discover Portal, type the discover target name as "inop01.citynext.com" or use its IP address to connect
    • Select the Targets tab; click the Connect button
  • Log on to apmgr01; create place holder for web services
    • Navigate to Administrative Tools, and then click Internet Information Services (IIS) Manager
    • Add 3 websites:
      • CSM
        • SiteName: CSMService
        • Application pool: CSMService
        • Physical path: E:\
        • Port: 8012
      • DSM
        • SiteName: DSMService
        • Application pool: DSMService
        • Physical path: F:\
        • Port: 8004
      • MDM
        • SiteName: MDMService
        • Application pool: MDMService
        • Physical path: G:\
        • Port: 8105
    • Change the identity to citynext\platsvcacct for the 3 app pools above
    • Set Full control for F: to the user “citynext\PlatSvcAcct”
  • Log on to apmgr01, apmgr02
    • On each cluster node, copy the VBScript 3 times to the path below and rename them:
      • C:\Windows\System32\inetsrv\FailoverCSM.vbs.
      • C:\Windows\System32\inetsrv\FailoverDSM.vbs.
      • C:\Windows\System32\inetsrv\FailoverMDM.vbs.
    • Edit each VBScript, change the SITENAME and APPPOOL_NAME variables to the real website name and app pool name
    • Confirm that the 3 VBScripts are copied into both VMs
  • Log on to apmgr01
    • Navigate to Administrative Tools, and then click Failover Cluster Manager; connect to the cluster
    • Roles -> right click -> configure role -> generic script -> script file path:
      • C:\Windows\System32\inetsrv\FailoverCSM.vbs.
      • C:\Windows\System32\inetsrv\FailoverDSM.vbs.
      • C:\Windows\System32\inetsrv\FailoverMDM.vbs.
    • Client Access Point
      • CSMService
      • DSMService
      • MDMService
    • Select Storage
      • For CSM: Cluster Disk X (Make sure this cluster disk represent E:\)
      • For DSM: Cluster Disk X (Make sure this cluster disk represent F:\)
      • For MDM: Cluster Disk X (Make sure this cluster disk represent G:\)
    • Click Next until completion
    • Cluster -> Roles -> Select CSMService -> Resources -> Server Name -> IP Address -> Properties -> Static IP Address -> 192.168.1.75
      • Do the same steps for DSMService, Static IP: 192.168.1.76
      • Do the same steps for MDMService, static IP: 192.168.1.77
    • Follow the steps below for the 3 roles:
      • Roles -> Select CSMService -> Resources -> Server Name -> right click -> Properties -> Policies -> Maximum restarts in the specified period = 0
      • Double click role -> failover -> Maximum failures in the specified period = 100
      • Do the same steps for DSMService and MDMService
    • Reset the properties and script properties for 3 roles, and bring the 3 roles online
5.9 Install IIS Failover 2.png
5.9 Install IIS Failover 3.png
  • Please move these roles to the second node, and make sure they are in a normal state
5.9 Install IIS Failover 4.png
5.9 Install IIS Failover 5.png
Back to top

Install SQL

  • Log on to dbsql01, dbsql02, inscom01
    • Install SQL Server 2012 Enterprise with all features
    • Important:
      • Install the SQL Server on the F Drive, and data should be stored under “F:\SQLDATA\MSSQL11.MSSQLSERVER\MSSQL\DATA”
      • Set the authentication mode as Mixed Mode, and password as “citynext0!”
      • Set citynext\infrsvcacct as the SQL server administrator group
      • Add current user to analysis service, distributed replay controller service
    • After SQL is installed, reset the MSSQLSERVER running account to citynext\infrsvcacct
  • Log on to inop01
    • Install SQL Server 2012 Enterprise with shared features only

Setup SQL AG

  • Log on to dbsql01, dbsql02
    • Install the failover clustering feature
      • Add the Failover Clustering feature to the VM. In the Initial Configuration Tasks window, click Add features.
      • In the Add Features Wizard, select Failover Clustering as shown below and click Next
      • In Confirm Installation Selections, click Install
      • When the Failover Clustering feature installation is completed, click Close
  • Log on to dbsql01
    • Open Server Manager -> Failover Cluster Manager
    • Click “Create cluster” on Actions list
    • Add “DBSQL01” to servers on the Select servers page
    • Click next until completion to finish the cluster validation test
    • Input Cluster Name as “SQLFC” on the Creating New Cluster page
    • Finish Create Cluster Wizard
    • Failover Cluster Manager -> Cluster Core Resources -> IP Address -> Properties -> Static IP Address -> 192.168.1.139, Network -> 192.168.1.128/26

5.11 Setup SQL AG failover cluster manager.png
  • Add the other nodes “DBSQL02”
  • Log on to dbsql01 and dbsql02
    • Open SQL Server Configuration Manager, and right click on the SQL Server (MSSQLSERVER) to open the properties window. Enable the AlwaysOn Availability Groups feature in SQL service properties.

5.11 Setup SQL AG.png
  • Confirm MSSQLSERVER running account is located in citynext\infrsvcacct
  • Restart services “SQL Server (MSSQLSERVER)”
  • Log on to dbsql01
    • Create a single-node cluster on one of the nodes (DBSQL01)
    • Bring the cluster name online (SQLFC)
    • Failover Cluster Manager -> Cluster Core Resources -> IP Address -> Properties -> Static IP Address -> 192.168.1.139, Network -> 192.168.1.128/26
    • Bring the cluster name online SQLFC
    • Add the other nodes (DBSQL02)
  • Use SSMS to connect MSSQLSERVER dbsql01, dbsql02
    • Add citynext\infrsvcacct as a sysadmin role to the default SQL Server instance
    • Create a machine account for both nodes; grant sysadmin privileges to both accounts for both nodes:
    • Grant sysadmin permission to NT AUTHORITY\SYSTEM for both nodes
  • Use SSMS to connect MSSQLSERVER dbsql01
    • Create a new database (MyDB1) on DBSQL01
    • Execute a full backup of the database
    • Create the availability group (SQLAG) with synchronous commit, automatic failover, and readable secondary replicas
      • Open the SSMS; select the Object Explorer; right-click AlwaysOn High Availability and click New Availability Group Wizard, as shown below

5.11 Setup SQL AG 2.png
  • On the Introduction page, click Next. On the Specify Availability Group Name page type SQLAG for the Availability group name, then click Next again.
  • On the Select Databases page select MyDB1 and click Next. The database meets the prerequisites for an availability group because you have executed at least one full backup on the intended primary replica.
  • On the Specify Replicas page, click Add Replica
  • The Connect to Server dialog pops up. Type DBSQL02 for the Server name, then click Connect.
  • Returning to the Specify Replicas page, you should now see DBSQL02 listed in Available Replicas. Configure the replicas as shown below. When you are finished, click Next.

5.11 Setup SQL AG 3.png
  • On the Select Initial Data Synchronization page, select Full and use the shared location "\\INOP01\SQLAGCache", click Next.

5.11 Setup SQL AG 4.png
  • On the Validation page, click Next. This page should look similar to the image below. There is a warning for the listener configuration because you have not yet configured an availability group listener. Ignore this warning for right now.

5.11 Setup SQL AG 5.png
  • On the Summary page, click Finish, then wait while the wizard configures the new availability group. On the Progress page, you can click More details to view the detailed progress. Once the wizard is finished, inspect the Results page to verify that the availability group is successfully created, as shown below, then click Close to exit the wizard.

5.11 Setup SQL AG 6.png
  • In Object Explorer, expand AlwaysOn High Availability, then expand Availability Groups. You should now see the new availability group in this container. Right-click DBSQL01 (Primary) and click Show Dashboard. Your AlwaysOn Dashboard should look similar to the one shown below. You should be able to see the replicas, the failover mode for each replica, and the synchronization state.
  • From the Start menu, open Server Manager again, then expand Features, then expand Failover Cluster Manager, then expand SQLFC.Citynext.com, and then expand Services and applications. A new clustered service called SQLAG has been created.
  • Open failover cluster manager -> roles -> SQL AG Role -> right click -> properties -> failover -> maximum failure in the specified period = 100, period = 1

5.11 Setup SQL AG 7.png
  • Log in to the Azure management portal
    • In the Azure portal, navigate to each VM hosting a replica and view the details
    • Click the Endpoints tab for DBSQL01
    • Add a stand-alone endpoint, use the name "SQLAG1433", the public and private port should be set to 1433
    • Enable "Create A Load-Balanced Set" and "Enable direct Server Return"
    • Set the Load-Balanced Set name as "SQLAG1433-ENLB", Probe Port set to 59999, click ok
    • Click the Endpoints tab for DBSQL02; add an endpoint to an existing Load-Balanced Set; select SQLAG1433-ENLB" and input the name “SQLAG1433”
    • Select “SQLAG1433” in the endpoints list; click the button “MAMAGE ACL”
    • Create some ACL rules that allows all Cloud Service Public IP addresses (please update the IP addresses in the red box) and deny any others, please refer to the image below:

5.11 Setup SQL AG ACL details.png
  • Log on to DBSQL01 to create the availability group listener
    • Open the SSMS; select the AlwaysOn High Availability; right-click Availability Group Listener and click Add Listener
      • Type the listener name as "SQLListener", set the port to “1433”
      • Select “Static IP” as the Network Mode and add the static IP 192.168.1.190
      • Open PowerShell as an administrator and then run the following script to change the listener IP address (replace the highlighted variables)
5.11 Setup SQL AG availability listener group script.png
5.11 Setup SQL AG availability listener group image.png
  • Change host name with SQLListener in the DNS Server
    • Log on to INAD01, and select the “DNS”
    • Open the tree view: DNS -> INAD01 -> Forward Lookup Zones -> citynxt.com
    • Select the host name “SQLListener”
    • Change the IP address which is the same as the Azure cloud service Public Victual IP Address for XXDBSQL.cloudapp.net
  • Log on to dbsql01 to update Readable Secondary to “Yes”
    • Use SSMS to connect MSSQLSERVER dbsql01
    • Click AlwaysOn High Availability -> Availability Groups -> SQLAG(Primary)
    • Right click SQLAG(Primary) and select properties
    • Update Readable Secondary to “Yes”

5.11 Setup SQL AG failover cluster manager 2.png

For details, visit the following Microsoft website: http://msdn.microsoft.com/en-us/library/dn249504.aspx#BKMK_Create_Cluster

Back to top

Install HDP 2.1

  • Log on to DBHDP01, 02, 03, 04, 05, and follow the steps below for each machine:
    • Download the installation file “HDP 2.1.3.0.zip” from: http://hortonworks.com/products/releases/hdp-2-1/, and save the path as “C:\Deploy\tools\ hdp-2.1”
    • Download and install JAVA and JDK 7.55 x64; confirm that the installation path is “C:\Java\jdk1.7.0_55\” and “C:\Java\jre7\”
    • Download and install python 2.7.6 x64, the install path is “C:\Python27”
    • Download VC++ 2010 from: http://www.microsoft.com/en-us/download/details.aspx?id=14632 and install it
    • Copy AutoStartHDP.xml and clusterproperties.txt from the release folder to c:\deploy\tools\hdp-2.1
5.12 Install HDP 2.1.png
  • Clear the content for “clusterproperties.txt” and replace it with the text below; the file location is c:\Deploy\tools\hdp-2.1
5.12 Install HDP 2.1 2.png
  • Copy deploy scripts from Release to c:\deploy\scripts
  • Run Windows PowerShell ISE as the administrator
    • C:\Depoly\scripts\DeployHDP21.ps1

HDP 2.1 Advanced Configuration (Optional)

Enabling Tez can significantly improve the performance of the data query service. However, it is only applied when encountering the scenario below:
  • 1:1 binding between entity schema and object schema
    • Confirm that each entity schema only has one bound object schema. Otherwise, skip this section.

Steps:
  • Log on to dbhdp01
  • Change F:\hdp\hadoop\hive-0.13.0.2.1.1.0-1621\conf\hive-site.xml
5.13 HDP 2.1 Advanced Configuration (Optional).png
  • Change F:\hdp\hadoop\tez-0.4.0.2.1.1.0-1621\conf\Tez-Site.xml
5.13 HDP 2.1 Advanced Configuration (Optional) 2.png
  • Change F:\hdp\hadoop\hadoop-2.4.0.2.1.1.0-1621\etc\hadoop\capacity-scheduler.xml
5.13 HDP 2.1 Advanced Configuration (Optional) 3.png

Install Hive ODBC

  • Log on to APMGR01, APMGR02; APSVC01; APSVC02; DBHDP01; INMGR01; INMGR02
  • Download the latest Hive ODBC X64 1.4.8.1008 from http://hortonworks.com/hdp/addons/
  • Install “HortonworkHiveODBC64.exe” to a default location
  • Open ODBC Data Sources (64bit), click System DSN - > Add -> “Hortnworks ODBC Driver” to add Hive ODBC, refer to the image below:
5.13 Install Hive ODBC.png
  • Click test for results

Install SCOM

  • Log on to INSCOM01
  • Install System Center Operation Manager 2012
    • In select features to install page, select all except “Reporting server”
    • In Specify an installation option page, select “Create the first Management server in a new management group” and input the management group name as “CITYNEXTSCOMGrp”
    • In Configure the operational database page, input the server name and the instance name as “inscom01\mssqlserver”
    • In Configure the data warehouse page, input the server name and the instance name as “inscom01\mssqlserver”
    • In Configure Operations Manager accounts page, all accounts should use the user “citynext\infrsvcacct” and password “citynext0!”
    • Please use the default value for the other pages

Setup Static Routing

  • Add route in the Azure server: APMGR01 APMGR02
    • Log on to APMGR01 and APMGR02; open a command prompt as the administrator; execute the following command:
Route add <SQLlistener ip> mask 255.255.255.255 <VM Gateway address> metric 5
< SQLListener IP> is same as the XXDBSQL cloud service public IP in Azure management portal
< VM Gateway address> Please running command “ipconfig /all” in APMGR01 to get the VM gateway address

Install Excel 2013 and FEDI Excel Permission

  • Log on to FEDI01 and FEDI02
  • Install Excel 2013
  • Create directory "C:\Windows\SysWOW64\config\systemprofile\Desktop” (for 64 bit Windows) or "C:\Windows\System32\config\systemprofile\Desktop” (for 32 bit Windows).
  • Set full control permissions for directory desktop to the user “citynext\PlatSvcAcct”
  • You will need to log on to the machine using the service's logon account to run the file at least one time. It will make sure Excel was activated
  • Add account “citynext\PlatSvcAcct” to local administrator group

Configure WSUS (Optional)

Install SQL Backup Plan

  • Log on to dbsql01 and dbsql02, and do the following for both machines:
    • Grant permission to Integration Service for platadmin01
    • Open SQL Studio; select "Integration Service"; connect to dbsql01
    • Under DBSQL01 -> Stored Packages -> MSDB -> Maintenance Plans, right click, import backup plan packages
      • For dbsql01, Import 2 packages from release share\deploy\scripts\databackup\dbsql01
      • For dbsql02, Import 2 packages from release share\deploy\scripts\databackup\dbsql02
    • In SSMS, SQLAG -> Property, set the backup preference to "primary only"
Follow the steps below for both DBSQL01 and DBSQL02:
  • Log on to DBSQL01/DBSQL02
  • Open the services panel; set the SQL Server Agent (MSSQLSERVER) to start automatically; start it
  • Open SSMS -> Management -> Maintenance Plans
    • Double click “FullBackup”
    • Double click “Back Up Database (Full)”
    • Select “For availability databases, ignore Replica Priority for Backup and Backup on Primary Settings”
5.19 Install SQL Backup Plan.png
  • Click OK
  • Confirm that there is a SQL agent job “FullBackup.Subplan_1” automatically generated under the SQL Server Agent
5.19 Install SQL Backup Plan 2.png
  • Double click “FullBackup”
  • Double click “Back Up Database (Full)”
  • Uncheck “For availability databases, ignore Replica Priority for Backup and Backup on Primary Settings”
  • Click OK
  • Open SSMS -> Management -> Maintenance Plans
    • Double click “DiffBackup”
    • Double click “Back Up Database (Differential)”
    • Select “For availability databases, ignore Replica Priority for Backup and Backup on Primary Settings”
    • Click OK
    • Double click “Backup DataBase Task”
    • Select “For availability databases, ignore Replica Priority for Backup and Backup on Primary Settings”
    • Click OK
5.19 Install SQL Backup Plan 3.png
  • Confirm that there is a SQL agent job “DiffBackup.Subplan_1” automatically generated under the SQL Server Agent
5.19 Install SQL Backup Plan 4.png
  • Double click “DiffBackup”
  • Double click “Back Up Database (Differential)”
  • Uncheck “For availability databases, ignore Replica Priority for Backup and Backup on Primary Settings”
  • Click OK
  • Double click “Backup DataBase Task”
  • Uncheck “For availability databases, ignore Replica Priority for Backup and Backup on Primary Settings”
  • Click OK
Back to top

Install Anti-virus (Optional)

Note: In this chapter, the default anti-virus tool is provided. You can choose any proper anti-virus utility to install.
  • Log on to each machine
  • Install “Microsoft System Center Endpoint Protection (Forefront)”
  • Update the virus and spyware definitions
  • Set a proper daily scan schedule

Install Test Agent

Miscellaneous

  • Log on to inmgr01, inmgr2
    • Add platsvcacct into below group
      • Start Menu > Administrative Tools > Local Security Policy > Local Policies > User Rights Assignment > Log On As Batch Job
  • Log on to inop01
    • Copy release builds from release share to c:\deploy\builds
    • Copy release scripts from release share to c:\deploy\scripts
    • Share c:\deloy as \\inop01\deploy with everyone default permission
  • Log on to dbsql01
    • Copy release scripts from release share to c:\deploy\scripts
    • Administrator PowerShell
      • cd c:\deploy\scripts
      • .\deploy.ps1 –source \\inop01\deploy\builds
      • .\deploy.ps1 configurationservicedblocal
      • .\deploy.ps1 dsmdblocal
      • .\deploy.ps1 camindexdblocal
      • .\deploy.ps1 coredblocal
    • Open SSMS on dbsql01
      • Run c:\deploy\scripts\ExportCertAG.sql
  • Log on to dbsql02
    • Copy release scripts from release share to c:\deploy\scripts
    • Copy .cer and .pvk files (total 8 files) from dbsql01 c:\deploy\scripts to dbsql02 c:\deploy\scripts
    • Open SSMS on dbsql02
      • Run c:\deploy\scripts\ImportCertAG.sql
5.22 Miscellaneous.png
  • Log on to inop01
    • Copy release builds from release share to c:\deploy\builds
    • Copy release scripts from release share to c:\deploy\scripts
    • Share c:\deloy as \\inop01\deploy with everyone using the default permission
Back to top


Deploy Microsoft CityNext Big Data Solution Accelerator

Execute Deployment Script

  • Log on to inop01 with citynext\platadmin01
  • Open administrator PowerShell prompt
  • cd c:\deploy\scripts
  • .\deployRemote.ps1 –stage1
    • This script will install environment variable and performance counters first. All machines except inop01 will restart after installation.
    • Wait until all machines restart and begin initialization
  • .\deployRemote.ps1 –stage2
    • This script will install all CityNext components on to each machine
  • Edit c:\deploy\scripts\databackupremote.ps1, change AzureStorageUrl and AzureStorageKey to the current one
    • Log on to Azure management -> Storage -> you created storage group -> Containers page, click “ADD”
    • Create a new container with the name “backup”
    • Get the container URL in this page as AzureStorageUrl
    • Log on to Azure management -> Storage -> you created storage group
    • Click “Manage access Keys” on this page
    • Get the “Primary Access Key” in jump box as AzureStorageKey
  • Open task scheduler -> LogArchiveScheduler and DataBackupScheduler
    • Change “configure for” in the general tab to Windows Server 2012 R2
6.1 Execute Deployment Script.png
  • Change c:\deploy\tools\LogArchive\logarchive.exe.config, the placeholder #AzureStorageKey# and #AzureStorageUrl# to the current one
    • For private azure deployment; if you want to archive the log to a local path, use the sample below to replace this key, don’t change “{0}” in this sample. It will archive the log file to \\inop01\backup\log:
<add key ="CopyCommand" value ="{0} \\inop01\backup\log /s /y /I"/>
  • Log on to apsvc01, apsvc02
    • Open IIS -> ARR web site -> Authentication -> Windows Authentication -> Provider
    • Remove negotiate
    • Azure Deployment Only:
      • Edit Environment Variable, change “ConfigServiceUri” to http://localhost:8000/
  • Log on to fedi01 and fedi02
    • Set Full control for the folder “C:\CityNextBDP\svchost\DIPullController” to the user “citynext\PlatSvcAcct”
  • Log on to apsvc01, apsvc02
    • Grant “Users” with full permission in C:\CityNextBDP\svchost\CamAgent
  • Log on to dbhdp01
    • Open task scheduler, import the task from c:\deploy\scripts\Restart HiveServer2.xml

Verification

  • Log on to inop01
  • Open IE, go to http://inmgr:8100/RubberDuck
  • Click “New” on the bottom left, a set of verification BVTs will be launched
  • Wait until the progress is completed (lasts about 1 hour)
  • Click “View Result” on the bottom right
  • If 38 test cases are all green and the overall status shows “pass”, the deployment was successful
  • If not, click “View Detail” on the bottom right in order to retrieve the detail error log and debug case by case
Back to top


Deploy Operation Packages

Monitoring Data Deployment

HBase script

  • Join machine to CityNext domain DBHDP01
  • Open cmd and navigate to F:\hdp\hadoop\hbase-0.98.0.2.1.1.0-1621-hadoop2\bin
  • Input “hbase shell”
  • Open test script “SCOMMonitoringHbaseData.hs”
  • Copy everything to cmd then wait for the execution to be completed

SQL server script

  • Join machine to CityNext domain inop01
  • Open “SQL management studio”
  • Connect to “sqllistener” with admin account
  • Execute script “SCOMMonitoringSQLData.sql“

Configure SCOM

  • Make sure AD setup is ready
    • Confirm that all servers are in the domain
    • Confirm that all accounts are created
  • Confirm all monitoring data is already injected
  • Log on to the “inscom01” VM in the CityNext environment with citynext\platadmin01
  • Open the “Operation Console” of the SCOM

Agent deployment

Deploy

  • Click the “Administration” tab on the main page of the console
  • Select “Device Management”; right click; then choose “Discovery Wizard”
  • Select “Windows computers”; then click “Next”
  • Select “Automatic computer discovery”; then click “Next”
  • Select “Use selected Management Server Action Account”; then click “Discover”
    • Wait until scanning is completed
    • Check that the discovery result is the same as Infrastructure in the section above
  • Select “Select All”; then click “Next”
  • Select “Local System”; then click “Finish”
  • Wait until the push agent installation to the servers is completed

Verification

  • Click “Monitoring” in main page
  • Confirm that “Citynext Platform Monitoring” is present

Create Monitoring Account

Deploy

  • Click the “Administration” tab on the main page of the console
  • Select “Run As Configuration”; then right click
  • Click “Create Run As Account”
  • Select “Windows” in type
  • Input name as “Citynext Monitoring Account”; then click “Next”
  • Input citynext\platadmin01 from section above
  • Click “Create”

Verification

  • Go back to the “Administration” tab on main page of the console
  • Select “Accounts” in “Run As Configuration”
  • Check the “Type: Windows” list, the account should be listed there
Back to top

Import Solution Accelerator Management Package

Basic management package import

  • Copy all the content of the management package from the release folder to local
  • Click the “Administration” tab on the main page of the console
  • Select “Management Packs”
  • Right click; then select “Import Management Packs…”
  • Click “Add”; then select “Add from disk”; then add all that is listed in the table below:
7.3.1 Basic management Package import.png
  • Click “Install” and wait for the installation to be completed

CityNext solution accelerator management package import

Deploy

  • Confirm that the management package install was successful
  • Click the “Administration” tab on the main page of the console
  • Select “Management Packs”
  • Right click; then select “Import Management Packs…”
  • Click “Add”; then select “Add from disk”; then add all that is listed below
    • CityNext MP Monitoring
  • Click “Install” then waiting for installation done.

Verification

  • Click “Monitoring” on the main page
*Confirm that “ CitynextMP Monitoring” is listed

Back to top

Binding

Categorization

There will be different levels of binding operations depending on the consistency between the target environment and the default topology.
  • Exactly same infrastructure topology and configuration as shown in this document
    • Use “Web Application Transaction” monitoring binding
  • Same scale but different VM hostname
    • Use “Distribution Diagram binding” – “Host diagram”
    • Use “Web Application Transaction” monitoring binding
  • Scale out for more instances and a different VM hostname
    • Use “Distribution Diagram binding” – “Architecture diagram”
    • Use “Distribution Diagram binding” – “Component diagram”
    • Use “Distribution Diagram binding” – “Host diagram”
    • Use “Web Application Transaction” monitoring binding

Web application transaction monitoring binding

  • Click the “Authoring” tab
  • Expand “Management Pack Templates”
  • Choose “Web Application Transaction Monitoring”
  • Double click each item in the list below
  • Click “Configure settings” in the right section of the dialog
  • Choose “NTML” as the authentication method
  • Choose “Citynext Monitoring Account” User Account then click “OK”
  • Click “Apply” and wait for the results to be saved
7.4.2 Web Application Transaction monitoring binding.png

Verification

  • Select the “Monitoring” tab
  • Select “Web Application Transaction monitoring” View
    • If not show this view under “Monitoring”, you can click “Show or Hide Views…” to select “Web Application Transaction monitoring”
  • Double click “Web Application State” to check all items mentioned in 7.4.2 marked in green

Distribution diagram binding

Common process

  • Select the “Authoring” tab on the main page
  • Click “Distributed Applications”
  • Double click the specified target to edit
  • Click the “OK” button when the pop-up below is displayed and make the necessary changes accordingly:
7.4.3.1 Common process.png

Host diagram

  • Follow the list below and edit them one by one
7.4.3.2 Host diagram.png
  • Input “C:”,”D:” or “F:” into binding hard disk for the specified server (for recommended configuration only data node has F:)
  • Input “Windows Server 2012 R2” into the binding CPU, RAM and OS

Component diagram

  • Follow the list below and edit them one by one
  • If the node doesn’t contain an object, use the search box for the diagram name and input the result for the object with same name
7.4.3.3 Component diagram.png
7.4.3.3 Component diagram 2.png

Architecture diagram

  • Follow the list below and edit
7.4.3.4 Architecture diagram.png

Verification

  • Select the “Monitoring” tab
  • Select the “Citynext Monitoring” Management package
  • Click “Overall Status View”
  • Expand “citynext” by clicking “+”
  • Expand “DI.citynext” by clicking “+”
  • Expand “PullControllerWinSrv.DI.citynext” by clicking “+”
  • Expand ”N01.PullControllerWinSrv.DI.citynext” by clicking “+”
  • Expand “Hardware” by clicking “+”, then check all items marked with green
  • Expand “Software” by clicking “+”, then check all items marked with green
  • Expand “DependOn” by clicking “+”, then check all items marked with green
  • Expand “Depend N01.PullControllerWinSrv.DI.citynext ” by clicking “+”, then check all items marked with green
Back to top

SCOM Email

  • Create folder "C:\SCOM_SCRIPTS\Notifications" on inscom01
  • Copy the SCOMEnhanceEmailNotification.ps1 and config.xml into the folder from:
    • \Daily_CityNextMonitoring\1.0.1119.1\Release\Scripts
  • Create a live.com email account
  • Change the highlighted content in the config.xml with an email account name and password:
7.5 SCOM Email config script.png
  • Setting up the SCOM Command notification channel:
    • Open the SCOM Console; go to the administration table
    • Right click on “Channels” under the “Notifications” node
    • Select “New Channel”; then “Command”
    • Follow the subsequent steps in order to complete the creation of new command channel
7.5 SCOM Email.png
7.5 SCOM Email 2.png
  • Settings:
    • Full path of the command file: C:\Windows\System32\WindowsPowerShell\v1.0\Powershell.exe
    • Command line parameters: -Command “& ‘”C:\SCOM_SCRIPTS\Notifications \SCOMEnhancedEmailNotification.ps1″‘” -alertID ‘$Data/Context/DataItem/AlertId$’ -Recipients @(‘SCOM Admin;user@outlook.com, SCOM Admin2;user2@outlook.com’)
      • Change the boldface content using the real recipient mailboxes with the mailbox display name
    • Startup folder for the command line: C:\SCOM_SCRIPTS\Notifications
  • Right click on “Subscribers” and Select “New Subscribers”
  • Follow the subsequent steps in order to complete the creation of new subscribers
7.5 SCOM Email 3.jpg
7.5 SCOM Email 4.jpg
7.5 SCOM Email 5.jpg
  • Add and select the channel created in the previous step on this page
    • Right Click on “Subscriptions” and select “New Subscriptions”
    • Follow the subsequent steps to complete the creation of new subscriptions

7.5 SCOM Email 6.png
7.5 SCOM Email 6.png
7.5 SCOM Email 8.jpg
  • Add and select the subscribers created in the previous steps on this page

7.5 SCOM Email 9.jpg
  • Add and select the channel created in the previous steps on this page
Back to top

SCOM Daily Report

Deploy daily email engine

  • Log on to inscom01
    • Copy . from: \Daily_CityNextMonitoring\1.0.1111.1\Release\DailyMailNotification to F:\SCOM\DailyReport
  • Create a windows schedule task to run the CityNext.Monitoring.SCOMNotification.exe daily
    • Control Panel\System and Security\Administrative Tools\Task Scheduler
    • Action->Create Task -> name: “Daily SCOM Email”
    • Select “Run whether user is logged on or not” on the general page and input a task name
    • Create a recurring alert; set for 3 times a day
    • Start a program: F:\SCOM\DailyReport\CityNext.Monitoring.SCOMNotification.exe
    • Start in: F:\SCOM\DailyReport
    • Save the task

Verification

  • Log on to inscom01
  • Open task scheduler
  • Manually trigger a schedule for the “Daily SCOM Email”
  • After that is complete, check \\inop01\SCOMReportShare; there will be a .html file inside and a listing of the status for all monitors

Back to top

Last edited Jan 21, 2015 at 6:04 AM by gheadd, version 40