Application of full-media full-workflow management software in Beijing TV IP-based Master Control Room (MCR)

I. Overview

With the rapid development of the UHD video industry, 4K/8K UHD signals impose higher bandwidth requirements, causing traditional SDI (Serial Digital Interface) signal scheduling matrix systems to expand continuously. Cabling and signal processing chains have become extremely complex, the architecture does not support flexible expansion, and it is difficult to use software to reconfigure system workflows. Conventional SDI-based acquisition, production, scheduling, and distribution methods can no longer meet the demands of technological advancement. The industry consensus is to adopt an IP-based architecture built on ICT (Information and Communications Technology). On one hand, this addresses high-bandwidth signal transport and scheduling; on the other, IP-based networking overcomes the limitation that traditional broadcast television signals can only be transmitted linearly from front to back, providing the technical foundation for future media convergence, Cloud Computing, and big data analytics.

Adopting IP technology to build a new production and broadcast system can readily solve the problems that traditional systems cannot or struggle to address, while also bringing many additional advantages.

II. Background -- Building a Smart IP Master Control Room (MCR) System

The smart IP Master Control Room (MCR) system focuses on deep coordination among SDN network management, automation, intelligent emergency switching, monitoring, Resource Pool management, Information Publishing System, post-event Data Analysis, asset management, and other application modules. Combined with the station's operational requirements and workflows, and on the premise of ensuring safety and stability, it achieves intelligent innovation in IP MCR operations and management:

• Thoroughly analyze on-duty staff tasks in the equipment room, decompose and restructure work tasks according to security levels to achieve system intelligence.
• Intelligent equipment room monitoring and O&M: Comprehensive health monitoring of signals, devices, network, and environmental factors such as temperature and air conditioning. Alarm information is pushed to designated personnel in a timely manner according to predefined severity levels.
• Intelligent business-layer O&M: The O&M system can automatically execute tasks after entry and review, including automatic selection of links and Resource Pool devices and automatic parameter configuration. It provides comprehensive monitoring of the entire task workflow, with automatic emergency response upon signal or device failure, activation of redundant resources, and service migration. During automatic Task Execution, alarms and emergency operation information are pushed to designated personnel according to predefined severity levels, enabling remote monitoring and task execution review.
• AI-driven growth of the O&M system: Through comprehensive data collection, aggregation, and storage of O&M data, along with multi-dimensional Data Analysis, the system facilitates post-event fault tracing, causal chain deduction, and event correlation analysis. This in turn continuously optimizes the system's intelligent management and automated execution workflows.

III. Scenario Overview

The IP scheduling system handles reception, scheduling, and distribution of 8K/4K/HD signals via IP multicast streams. Centered on SDN, the system is equipped with IP signal routing and scheduling management software. It supports customizable hardware routing switching panels with flexible data workflow and stage editing, achieving a truly software-defined network for IP scheduling. Built on the ST-2110 signal standard, the system also supports ST 2022-6 signal scheduling, ST 2022-7 redundancy, and PTP generation and distribution compliant with IEEE1588 and SMPTE2059. The system can lock to the synchronization system's PTP precision clock, enabling unified scheduling and time synchronization across the entire domain, with full compatibility for both compressed and uncompressed audio/video signals.

From a network security perspective, in addition to the IP scheduling system, a separate IP scheduling and management system with comprehensive device and business management and Control capabilities has been built. The IP scheduling system adopts a Resource Pool design approach, categorizing devices such as IPGs, IP Multiviewers, and Encoding/Decoding equipment and deploying them centrally in the core equipment room. Combined with workflow-based automated management software, the system achieves resource pooling, enabling dynamic allocation, flexible reallocation, emergency switching, and intelligent management.

The IP scheduling system enables mutual conversion between SDI and IP signals, multi-format IP signal conversion, delayed playout, and other functions. Incoming SDI signals must be converted to IP signals through an SDI/IP gateway before entering the system for scheduling and processing. Encapsulation, de-encapsulation, and scheduling within the IP scheduling system support SMPTE 2110-compliant IP multicast streaming.

IV. Technical Solution

4.1 General Overview

As the central node for station-wide IP signals, the IP scheduling system connects to Beijing Radio and Television Station (BRTV)'s selected live studios, playout center, incoming signals, program encoding distribution and transmission systems, baseband scheduling systems, and ingest systems, providing IP scheduling and routing capabilities for BRTV.

While achieving interconnection with the existing baseband Master Control Room (MCR) matrix system, partial modifications to the current baseband MCR matrix are required per the station's specifications to ensure playout safety. Station-wide signal scheduling and management are enabled. Studios, OB Vans (outside broadcast vans), and other external systems must support the IP scheduling system's signal delivery method.

The IP scheduling and management system must deliver comprehensive capabilities in Control, management, and monitoring to enable business workflow automation and fully support and safeguard operations. The system includes but is not limited to modules for data collection, device Control, signal scheduling, real-time alarm analysis, alarm management, post-event Data Analysis, topology display, task management, Task Execution, link modeling, macro management, reservation scheduling, knowledge base management, and Information Publishing System.

The IP scheduling and management system shall provide comprehensive Control, management, and monitoring capabilities along with automated task workflow execution to reduce the scope of manual operations and lower error rates.

The IP scheduling and management system shall provide a visual reservation-scheduling solution where reservation distribution and conflicts are clearly visible, avoiding unnecessary issues caused by resource conflicts.

The IP scheduling and management system can rationally allocate device time utilization, prevent unnecessary emergencies, and maximize resource utilization: expensive resources are not overused, similar resources are not left idle, similar tasks can reuse resources, and routing recommendations are highly automated.

The IP scheduling and management system provides workflow automation and intelligence through intelligent link allocation based on resource pooling, resource capability classification, and resource weighting. It rationally allocates time utilization for each resource, eliminates conflicts, ensures availability, and improves resource and facility utilization. The system provides workflow automation and intelligence through intelligent link allocation based on resource pooling, resource capability classification, and resource weighting, rationally allocating time utilization, eliminating conflicts, ensuring availability, and improving resource and facility utilization.

Integrated Information Display

In this project, information from the three core sections can be presented in a centralized and unified manner on the large display.

As shown above, the center displays SDN and core service switch status monitoring. The left side shows network monitoring and equipment room monitoring. The right side shows comprehensive monitoring of the compression and uplink systems. The bottom displays current reservation Task Execution tasks, including real-time Master Control Room (MCR) and playout task information. The center shows the main link topology interface. The upper right presents key parameter dashboards, along with Equipment Room Diagram alarms, alarm lists, and current task signal link displays.

1. Resource Reservation, including

2. Scheduling

3. Playout Detection and Monitoring

4. Data Collection and Data Analysis

5. Northbound interface with standard RESTful API for service and alarm presentation

4.2 Network Architecture

The IP scheduling system adopts a primary/backup redundancy model, designed for current and future applications with provisions for expansion and upgrade. A Spine-Leaf network architecture is used, with multiple uplink load-balanced links between Leaf and Spine layers to fully utilize bandwidth. Non-blocking design between Spine and Leaf ensures normal scheduling operations; data traffic can be forwarded from either Leaf or Spine nodes. The scheduling system uses Spine switches as its core, with test systems connected to Leaf switches, and expansion interfaces are reserved. Real-time working status of all devices and signals within the IP scheduling system can be monitored and queried. Subsystems interacting with the IP scheduling system include: studios, the playout center, ingest systems, the baseband Master Control Room (MCR) matrix, satellite reception, and incoming signals.

4.3 Key Components and Functions

4.3.1 SDN Scheduling System

For the playout Master Control Room (MCR) signal scheduling system, interconnection with numerous subsystems is required. Therefore, many brands of IP endpoint devices may coexist under the scheduling system switches. These devices do not necessarily support IS-04/05, and even those that do require considerable effort each time a new device is added. Overall, traditional PIM+IGMP is not the ideal choice for playout-domain signal scheduling systems. NAT (Network Address Translation) architecture is a viable alternative: under this approach, the IP scheduling system switches use SDN multicast flow-table-based switching. The SDN Control system directly controls the switches, directing flow table switching and multicast forwarding, facilitating unified address planning, management, and monitoring for incoming and outgoing streams of different subsystems. This way, the IP scheduling system switches can directly ingest multicast streams from different subsystems and translate their addresses to pre-planned internal addresses, without concern for the original multicast addresses. The same applies to multicast streams sent to downstream units: after NAT at the IP scheduling system switch egress, streams are converted to the subsystem's planned multicast addresses before entering the downstream subsystem.

Generally, SDN network architecture is divided into five parts: SDN application services, northbound API (Application Programming Interface), SDN controller, southbound interface, and SDN data plane. SDN application services provide a user-friendly interface and configure and manage data plane forwarders by calling the SDN controller's northbound interface. The northbound interface is the open interface between the SDN controller and network applications, abstracting data plane resources and status information into unified open programming interfaces. The SDN controller is the brain of SDN. It provides different levels of programmability to upper-layer network applications through the northbound interface and performs unified configuration, management, and Control of the SDN data plane through the southbound interface. The southbound interface is the open interface between the SDN controller and the data plane. The SDN controller programs the data plane through the southbound interface to implement forwarding and other network behaviors. The SDN data plane includes both software-based and hardware-based data plane devices. Data plane devices receive instructions from the controller via the southbound interface and process network data accordingly. SDN data plane devices can also feed back network configuration and runtime status information to the controller through the southbound interface. In the SDN architecture, the Control plane is separated from the data plane. The data plane becomes more generalized and no longer requires Control logic for various network protocols. Device Control logic is instead implemented by upper-layer applications, enabling software-defined network functions.

1. Core Functions

Build a highly reliable production and broadcast system centered on IP switches for signal transmission, signal switching, and scheduling. The system fully leverages IP network characteristics to achieve IP-based transport and signal switching for video, audio, synchronization, intercom, and Control systems, creating an advanced production and broadcast platform with demonstrative and guiding significance for the future. All IP service stream scheduling is completed through SDN-controlled network switching equipment; the SDN controller does not need to Control edge media nodes, which directly receive IP service streams pushed by the network switching equipment. In the core scheduling scheme, network switching equipment must support multicast NAT functionality.

Through multicast NAT functionality, service streams entering the network switching equipment can be replicated and forwarded to the corresponding egress ports. Multicast address and port modifications are applied at the egress ports connected to edge media nodes, so the receiving edge media nodes do not need to be aware of stream switching.

2. Safety Design

The SDN controller adopts a cluster Active/Standby working mode with one primary and one secondary node. The primary node provides external services while the secondary node serves as backup. Heartbeat and distributed state sharing between cluster nodes enable data and state synchronization. The cluster software synchronizes data from the primary to the secondary node and designates the secondary as the working node when a failure occurs.

Both the primary Server and the backup run High Availability monitoring programs that exchange heartbeat messages to monitor each other's status. When the backup cannot receive heartbeat messages within a specified time, it takes over the primary Server's service IP and continues providing services. When the backup receives heartbeat messages from the primary again, it releases the service IP address, allowing the primary to resume cluster management.

To ensure normal system operation when the primary Server fails, the system implements synchronization and backup of load cluster configuration information between the primary and backup servers to maintain consistent system state.

The SDN software features flow processing policies. By deploying Meter tables to the switches, it reserves different bitrates for each stream within the system (automatically or manually) to ensure non-blocking operation. Flow management includes: flow name, attributes, port number, sending device, sending device IP, bandwidth, connection time, connection duration, instantaneous bandwidth, and connection status display. It supports flow creation, modification, and deletion. Bandwidth overflow alarms are triggered when the system detects SFP port bandwidth overflow during routing switching, preventing the switch.

4.3.2 Workflow Automation

With the development of IP technology and increasing workloads, standardized, process-driven, and automated solutions are needed to handle daily operational requirements and improve efficiency. The broadcast center previously had the following issues:

To address the pain points of the original system, multiple services/subsystems including reservation scheduling, Information Publishing System, Task Execution, and detection monitoring have been deployed based on the Unified Control Platform (iSwiftMedia).

The system provides a visual reservation-scheduling solution where reservation distribution and conflicts are clearly visible, avoiding unnecessary issues caused by resource conflicts. The system seamlessly integrates with other departments' APIs, enabling automatic program retrieval, automatic scheduling, and automatic parameter entry, reducing workload.

The system integrates reservation, approval, and parameter entry functions, eliminating issues such as high data latency and data inconsistency caused by cross-platform operations in the broadcast center. It enables process-driven and automated business workflows while providing post-event data statistical analysis facilities and services.

The Reservation System includes reservation and scheduling functions, both of which are means of pre-planning and requesting resource usage. Scheduling is a highly regular form of resource pre-allocation, while reservation is more ad-hoc with greater uncertainty. Scheduling generally corresponds to longer-term planning. Due to its regularity, scheduling tends to use batch imports with pre-set time windows; reservation is more flexible and informal, representing short-term resource requests.

Task Execution System, receives and displays task execution information scheduled from the reservation system in real time, issues reminders in advance, and duty personnel execute tasks according to the scheduled time, opening up signal transmission routes and cooperating with equipment and signal detection systems to monitor the working status of all equipment on the task route at any time to ensure correct signal transmission. When equipment fails or the matrix status is incorrect, it can issue sound and light alarms to remind staff to resolve faults. At the same time, it provides means to allow operators to manually adjust task routes according to actual conditions to ensure smooth signal paths.

1. The system has the following features

The reservation module is used to edit application forms and complete the approval workflow. Reservation is a means of pre-planning and requesting resource usage. After on-duty personnel enter the day's scheduling tasks into the reservation module, the task management and execution system automatically maps them into executable task orders based on the reservation module's entries. After manual confirmation, the system enters an automatic inspection workflow and automatically configures relevant device parameters for signal scheduling at the designated time. The task management and execution system receives and displays task information scheduled from the Reservation System in real time, issues advance notifications, and allows on-duty personnel to manage, confirm, and execute task plans to establish signal routing paths. In coordination with device and signal detection systems, it continuously monitors the working status of all devices along the task route to ensure correct signal transmission. When a device fails or a Matrix state error occurs, audio-visual alarms are triggered to alert staff to resolve the issue. The system also provides means for operators to manually adjust task routing based on actual conditions to ensure uninterrupted signal paths.

An end-to-end, full-workflow automated intelligent platform. Using big data analytics and other techniques, it aggregates, stores, and processes data from multiple sources in real time, performs trend analysis, and provides recommendations to improve the operational quality of broadcasting institutions and facilities. The platform adopts a layered architecture: the upper layer contains the reservation and scheduling subsystems; the middle layer contains the task management execution and resource allocation units, along with system status monitoring modules; the lower layer contains the Monitoring System, signal scheduling system, and others. The system supports devices/subsystems from different Manufacturers, with different protocols and functions, enabling monitoring and Control of complex device clusters under a unified platform. After pooling complex device clusters, fast lookup and automatic link allocation can be performed based on resource weighting and capabilities. The system provides intelligent solutions for different scenarios, such as intelligent link pathfinding and intelligent replacement resource recommendations. The result is simple, intuitive, easy-to-use operation with higher resource utilization, greater system visibility, and comprehensive service assurance.

4.3.3 Network Management System

Advantages:

• B/S architecture, enabling users to access the system anytime, anywhere;
• Comprehensive solution;
• High degree of automation;
• Excellent elastic scaling capability;
• Microservice architecture for rapid adaptation to customized business workflows;
• Multi-level, multi-source data collection and aggregation capabilities, driving continuous operational optimization through big data analytics;
• Combined with Data Analysis and processing, provides operational reports from different perspectives to enhance insight into system operations;

Supports secondary proxies, device polling proxies, and clusters. Horizontal scaling through additional proxy instances supports greater system loads.

Based on a microservice architecture, it adapts to different interface types, protocol specifications, device types, and access media. Key features:

Software Solutions

For alarms and exceptions, quick access to the knowledge base via the software interface provides the following information and assistance:

Post-event data processing with exportable reports from multiple perspectives

Post-Event Data Processing Platform - Parameter Time-History Curves

Post-event query of alarm history with diverse query conditions - New Interface

1. Suitable for collection, aggregation, storage, and online query processing of massive data from various sources;

2. Collects data from different levels for operational data accumulation; supports data mining;

3. Provides data-driven recommendations and suggestions to drive continuous process and practice optimization;

Post-Event Data Processing Platform

Post-event query of historical data with composite display of multiple parameter groups

Post-event query of historical data, time-history curves

• Enterprise-grade post-event data analysis and processing platform;
• Suitable for collection, aggregation, storage, and online query processing of massive data from various sources;
• Leverages NoSQL databases such as Redis and MongoDB for superior performance and availability;
• Supports clustering and horizontal scaling with parallel processing;
• Online report generation;
• B/S architecture with multi-user concurrent access support;
• Simple, easy to maintain, easy to deploy;
• Supports high-speed batch data processing;
• Provides a GUI for interactive data export;
• Supports export in multiple file formats including XML, JSON, MS Excel, and CSV;
• Supports multi-source data aggregation to generate unified views and reports.