The Microsoft CityNext Big Data Solution Accelerator

The Microsoft CityNext Big Data Solution Accelerator is middleware that hosts big data-enabled smart city solutions on Microsoft’s cloud platform. It aims to solve key big data issues for smart city scenarios, as well as providing services for data ingestion, data modeling, city analytics, service integration and information dissemination. With the immeasurable amount of big data that is essentially in limbo and within grasp, there is a level of uncertainty regarding the actual serviceability of that data. The previous method in which to use data was based on the notion that what was being ingested had already been characterized, sent to a preset domain and catered to a predefined scenario. However, the Microsoft CityNext Big Data Solution Accelerator has been designed to surpass that type of archaic thinking, so that the various nonspecific data that is ingested –whether using old SQL, no SQL or new SQL to collect, modify and manipulate the data- can be distributed across multiple domains and utilized to combat a plethora of generated and unscripted scenarios.
Microsoft CityNext Big Data Solution Accelerator.png
The Microsoft CityNext Big Data Solution Accelerator offers the following key value propositions to cities:
  • The Microsoft CityNext Big Data Solution Accelerator is built on a modular architecture that allows cities to build smart city solutions with minimal development and quick deployment. Building and deploying a big data architecture seems daunting to even the best IT organizations, and “doing more with less” is a key imperative for cities in their budget-constrained environment. Using a modular architecture that consists of five key modules, the Microsoft CityNext Big Data Solution Accelerator effectively reduces the complexity involved in building big data solutions, and can significantly reduce the investment needed while enabling cities to bring innovative new solutions to their citizens.
  • The Microsoft CityNext Big Data Solution Accelerator provides a common framework that allows data and services to be shared across multiple departments and domains. This helps a city to truly utilize the power of big data to perform cross-domain data analysis, and to offer integrated services across the entire set of city functions. The scalability and versatility of the solution accelerator ensures that it can meet the needs of a wide range of city services and handle data in various forms –from operational systems, sensor networks, as well as social media. The cross-domain nature of the solution accelerator enables cross-agency collaboration and helps uncover insights that lead to positive changes to city services.
  • The Microsoft CityNext Big Data Solution Accelerator is built on Microsoft technologies and products that have consistently endured the test of the marketplace; offering the performance, reliability, and security that a smart city seeks. Together with our partners, we aim to develop the best-of-breed big data ecosystem with enhanced solutions for city big data, city analytics, and city service integration. Through customer POCs (proofs-of-concept), the solution accelerator has demonstrated how cities around the world can utilize cloud computing and big data technologies to ingest, manage and analyze data, disseminate information, and offer a wide range of big data powered services effectively.

With this solution accelerator, a variety of mechanisms are specifically orchestrated to control the overall flow of data so that the end-to-end devices and services capabilities are functioning at the highest levels of efficiency. The Microsoft CityNext Big Data Solution Accelerator connects the dots of the flow of data between the cloud, mobile devices, social media and big data. This essentially cultivates an environment which ensures that valued and vital information is constantly available to every important contributor within the overall city infrastructure so that the notion of “one city” is not just an idea, but a living reality.

Significance and Importance - The Microsoft CityNext Big Data Solution Accelerator uses various methods during the data collection process to consolidate city data from the eight city domains in order to store it in databases. No longer do city employees have to worry about the troubling and exhausting process of going through multiple channels in order to access city data to make decisions.
Frequently, city employees need to address issues that affect multiple functions of the city in real-time. In order for city-wide data to flow across an infrastructure, a solution accelerator must be put into place so that the data can be easily accessible to various domains. A variety of constraints, as well as administrative complexity, can be crippling and can hinder a city’s government from being transparent and accessible.
By unlocking value from data, Microsoft CityNext formulates a new way to optimize, standardize, and sustain an underlying innovative solution accelerator that all city functions can use. Whether the data is structured or unstructured, in the size of gigabytes to petabytes, or at rest or in motion, the Microsoft CityNext Big Data Solution Accelerator will manage countless amounts of city data and disseminate that data accordingly.
Microsoft CityNext Big Data Solution Accelerator b.png

Data Flow Process - The Microsoft CityNext Big Data Solution Accelerator is designed to ingest large volumes of city data, current and historical, and also outside data generated from citizens through social media. It is made up of five main components: data ingestion, city artifacts management, city analytics, city services management and info dissemination.
Initially, the data is acquired from streaming or batch data through the sensor network within the city infrastructure -whether it be through messages, databases or other relevant city files- from city services and from citizens through various clouds (private, hybrid or MS public) and social media networks. Once the data is retrieved, it is distributed to either push or pull channels and routed to the formatter so that the data can be consolidated into the operational data store, or sent directly to the real-time analytics runtime manger based on urgency. After the data is stored, it is analyzed and processed within the repository portal in order to determine through which service the data should be published for rendering.
By using the CityNext Analytics Repository Portal, a hub is created that allows developers and researchers the freedom to publish, share and discover city analytics modules and algorithms, and to connect these modules to specified CityNext data; as well as CityNext analytics-related datasets with predefined data models, for future analysis and subsequent rendering. City researchers and developers also have the opportunity to train and develop their modules and algorithms using a batch processing engine with connected CityNext predefined models. They can also develop and implement their city analytics services using asserts from the CityNext analytic repository.
According to the parameters set, the incoming data can provide batch analysis using city analytics modules and algorithms by Hadoop for unstructured data, and by a pre-built BI data warehouse for structured data with a batch processing engine. Real-time analysis using built-in CEP modules and algorithms through framework based on city streaming data with a real-time processing engine can also be delivered.
Subsequently, in order for the data to be further distributed, based on the nature of the content, the data is rerouted to either the batch-based or real-time analytics runtime manager. Finally, through the service determined by the analysis, data is disseminated through web coverage, a data portal or notification services. The notification services are primarily rendered via notifications and alerts, while web coverage is released mostly based on browsing and queries.
*Note: For further detailed information regarding the solution accelerator and how it is aligned with Microsoft CityNext please refer to High-Level Design Document.

Although the solution accelerator is functionally still a work in progress, we have instructional guides readily available that thoroughly expound upon the schematics, structure and design of the Microsoft CityNext Big Data Solution Accelerator. These guides will serve as invaluable tools to assist specialists, administrators and operators as it pertains to the operation and maintenance (Operation and Monitoring Guide), as well as the configuration and administration (Configuration and Administration Guide) of the solution accelerator.

In order to explore the nuances of the solution accelerator, we have included the Developer Guide so that developers can also have information at their fingertips. Also available is the Management Studio User Guide, which provides information regarding the developing tool that gives developers the ability to manage big data and schema for the solution accelerator. Most importantly, we haven't forgotten the user. The User Experience Guide is just as accessible as the other instructional guides because we realize how vital it is for our users to be included and in the know.

In addition to the guides mentioned previously, we are offering two separate demos of the Microsoft CityNext Big Data Solution Accelerator General Availability release. To showcase a more powerful and innovative functional version of the solution accelerator with samples, Microsoft CityNext Big Data Solution Accelerator V1 General Availability is available. For a an extended and in-depth look into the architectural structure and potential of the solution accelerator, the SDK edition (Microsoft CityNext Big Data Solution Accelerator V1 General Availability - SDK) is offered as an exclusive version that is specifically geared towards developers.

To get started testing the demos and samples that we have available, download them from https://mscitynextbigdata.codeplex.com/releases/view/125067. For a complete listing of all the documents relating to the Microsoft CityNext Big Data Solution Accelerator, click Documentation.

Back to top

Last edited Nov 24, 2014 at 7:44 AM by gheadd, version 12