cool hit counter Understanding the technical challenges of real-time communication using the design of the network access layer of an online game server as an example_Intefrankly

Understanding the technical challenges of real-time communication using the design of the network access layer of an online game server as an example


This article references and quotes part of the Tencent Game Academy's related technical article content, thanks to the original author for sharing.

1. Preamble

In terms of mainstream instant messaging applications, a complete instant messaging IM application is actually a set of network communication systems that combine two technologies: Instant messaging (IM = Instant messaging) and Real-time communication (RTC = Real-time communication). The reason why the abbreviation IM is used to refer to the whole instant messaging software is actually historical (because early instant messengers such as ICQ, that is, text chat, did not add real-time audio and video such as real-time communication technology), interested in this topic can look up the history of the development of IM on the Internet.

In the case of complete instant messaging applications like WeChat and QQ, going back to the essence of the tool, it consists of two main applications and technologies.

1) text chat in the broad sense: that is, the delivery of various chat messages as I most often understand them, and the technical implementation of this part is the much talked about IM communication (i.e. Instant messaging). 2) Real-time audio and video chat: including voice calls and video chat, the technical implementation of this part, from the perspective of network communication, is real-time communication (i.e. Real-time communication).

Let us recall. A few years ago, the mainstream mobile IMs on the market - such as WeChat, QQ, the now ad-filled NetEase, the half-dead Xiaomi MiChat, the already-dead Ali Lailai, and the errant Stranger - basically didn't have or added real-time audio and video chatting very late (let's put aside technical factors), not because they didn't want to do it, but because real-time audio and video, a real-time communication technology, does have a considerable threshold and is not easy to do.

So. For many IM application developers within the instant messaging network community, real-time communication technology is so important that in-depth research and understanding of the principles, technical practices of real-time communication technology can be of great benefit to the development of their own IM products.

In this paper, we will attempt to take a developer's perspective. We will sort out the various technical challenges faced in the process of developing the network access layer of the online game server, and provide the corresponding real-time communication network access layer solution ideas in a targeted manner, hoping that developers of instant messaging applications can get some inspiration from them.

Learning Exchange.

- Instant Messaging Development Exchange3 crowd:185926912[ recommend] - Introductory article on IM development for mobile: the One article is enough for newbies: developing mobile IM from scratch

(This article was simultaneously posted on.http://www.52im.net/thread-1915-1-1.html

2. Related articles

Computer Network Communication Protocol Relationship Diagram (Chinese Collector's Edition)》 《 High Performance Network Programming (I): How many concurrent TCP connections can a single server actually have》 《High Performance Network Programming (II): last decade, the famous C10K concurrent connection problem》 《 High Performance Network Programming (III): time to think about C10M concurrency in the next decade》 《 High Performance Network Programming (IV): a theoretical exploration of high performance network applications from C10K to C10M》 《 The Untold Story of Network Programming (VI): Understanding the UDP protocol in depth and using it well》 《 Lesser Known Network Programming (VII): How to make unreliable UDP reliable?》 《 Network programming for lazy people (3): a quick understanding of the TCP protocol one is enough》 《 A Lazy Introduction to Network Programming (IV): A Quick Understanding of the Differences Between TCP and UDP》 《 A Lazy Introduction to Network Programming (5): a quick understanding of why UDP is sometimes said to be more advantageous than TCP》 《 Technical Literacy: Next Generation UDP-based Transport Layer Protocol for Low Latency Networks - QUIC Explained》 《 Introduction to brain-dead network programming (I): follow the animation to learn TCP three handshakes and four waves》 《 Introduction to brain-dead network programming (II): what exactly are we reading and writing when we read and write to sockets?》 《 Seven Niu cloud technology sharing: using QUIC protocol to achieve real-time video live 0 lag!》 《 Understanding Latency Issues in Real-Time Audio and Video Chat One article is enough》 《 An introductory outline of real-time audio and video technology written for the uninitiated

3、The principle of network communication architecture of mainstream online games

Wikipedia definition of online games.

That is, a form of entertainment that allows multiple players to play online at the same time by connecting a dedicated server to the user's client device (mobile phone, PC, game console, etc.) through a computer network.

It follows that the online game involves three players.

1) Client side.

2) Network.

3) Servers.

In terms of network architecture, online games can be divided into.

1) C/S architecture: this is best understood.

2) P2P architecture: refers specifically to direct inter-client communication.

3) C/M architecture: in practice this is a hybrid of C/S and P2P architecture.

A typical network architecture is shown in the following figure.

P2P architectures are beyond the scope of this paper.

C/M architecture is similar to C/S architecture, and similar to the classic LAMP website architecture, the backend of a general C/S architecture game can also be divided into three layers as follows.

1) Network access layer.

2) The game logic layer.

3) Data storage layer.

The general C/S architecture of the game backend is layered, as shown below.

The problem domains and corresponding technology stacks faced by each of the network access, game logic, and data storage layers are very different, so making this division not only helps decouple modules, divide technology, and reuse components, but also facilitates the deployment of service operations and maintenance. It is this network access layer that will be discussed in this paper.

4、 Off-topic. What to understandC/M infrastructure?

Some may have questions about the C/M architecture in the previous section, how exactly does this architecture work in online gaming?

In fact, the synchronization mechanism is used in MMOs to ensure the consistency of the game world across clients.

There are two mainstream online game data synchronization mechanisms

1) State synchronization: i.e., the client is responsible for sending the player's operation to the central node (server or master client), which is responsible for calculating the game logic and broadcasting the calculation results to the client, which is then responsible for rendering the game results

2) Frame synchronization: the theory behind frame synchronization is that the game logic is driven by operation instructions, and as long as the sequence of operations is consistent, then the game results should be consistent.

That is, whichever data synchronization mechanism is used, it is actually the C/M architecture (i.e., Client/Master architecture) that is applied.

5. Role of the network access layer

The main tasks of the network access layer are.

1) establishing channels between the client and the backend service and between the clients,.

2) Receive a large number of concurrent requests from clients.

The key performance indicators for the assessment of the layer are.

1) High throughput.

2) Low latency.

Network access layer development thus tests the developer's high-performance network programming skills, i.e., the ability to solve C10K or even C10M.

Off-topic. For more information on the C10K and C10M topics of high-performance network programming, please read the following articles

High Performance Network Programming (I): How many concurrent TCP connections can a single server actually have》 《High Performance Network Programming (II): last decade, the famous C10K concurrent connection problem》 《 High Performance Network Programming (III): time to think about C10M concurrency in the next decade》 《 High Performance Network Programming (IV): a theoretical exploration of high performance network applications from C10K to C10M

6. Communication protocol selection for network access layer

Based on the OSI's seven-layer network reference model, we can divide the online gaming network into seven layers as well, as follows.

The first issue that developers face in the actual development process is whether to use TCP or UDP for the transport layer, and the following table briefly compares the advantages and disadvantages of the two. Combining the advantages and disadvantages of both, simply put, unless there are extreme latency requirements (e.g. FPS, MOBA games) that require UDP, TCP can handle most games.

In the actual game development whether using TCP or UDP approach, are less utilized through Socket programming directly, one because of the development workload, quality performance is difficult to ensure; second, platform compatibility is not good (for example, H5 does not provide socket programming capabilities), but based on higher-level communication protocols such as TCP-based HTTP, Websocket protocol, GRPC, and UDP-based implementation of QUIC, WebRTC protocol, etc..

A brief comparison of TCP, UDP protocols.

For a detailed comparison article on TCP and UDP protocols, you can briefly read the following information.

A Lazy Introduction to Network Programming (IV): A Quick Understanding of the Differences Between TCP and UDP》 《 A Lazy Introduction to Network Programming (5): a quick understanding of why UDP is sometimes said to be more advantageous than TCP》 《 Briefly describe the differences between the transport layer protocols TCP and UDP》 《 Why does QQ use the UDP protocol instead of TCP?》 《 Instant messaging protocol choice for mobile: UDP or TCP?

It is worth noting that based on security considerations, the browser standard does not provide UDP sending and receiving capabilities, QUIC protocol is only supported in chrome, WebRTC is not yet a browser de facto standard and the initial purpose of the protocol is to achieve point-to-point audio and video communication, the protocol content is too complicated not easy to refine the application of game development, so at this stage H5 games can only use HTTP or Websocket way communication.

Knowledge Literacy.

1) Regarding the QUIC protocol. Technical Literacy: Next Generation UDP-based Transport Layer Protocol for Low Latency Networks - QUIC Explained》;

2) Regarding WebRTC. The state of open source real-time audio and video technology WebRTC》、《 A brief description of the advantages and disadvantages of WebRTC, an open source real-time audio and video technology》、《 Interview with the father of the WebRTC standard: the past, present and future of WebRTC》;

3) Regarding Websocket. Beginner's Guide: The most complete explanation of the principles of web-based instant messaging technology ever》、《 Web-side Instant Messaging Technology Inventory: Short Polling, Comet, Websocket, SSE》、《 Quick Start for Beginners: A Concise Tutorial on WebSocket》、《 WebSocket Detail (I): Initial Understanding of WebSocket Technology》。

In addition, after the communication protocol is determined, the next thing to consider is the serialization of the game object, serialization is mainly text-based, binary-based two, their advantages and disadvantages are shown in the following table. In the development process, text serialization is generally used first to facilitate front-end and back-end development interfacing, and then switched to binary serialization to reduce transmission traffic and improve coding and decoding efficiency before the game is officially launched.

Main serialization methods for game objects.

For more information on Protobuf, see.

Protobuf communication protocol in detail: code demo, detailed principles, etc.》 《 Protobuf is strongly recommended as a data transfer format for your instant messaging application》 《 The Kingfisher Handbook team shares: still using JSON? Protobuf makes data transfer more economical and faster (principle chapter)》 《 The Kingfisher Handbook team shares: still using JSON? Protobuf makes data transfer more economical and faster (in practice)

As for data security issues, to protect sensitive data security developers can choose secure https or WSS communication protocols, while for direct TCP protocol-based communication, RSA can be used to negotiate the encryption secret key first, and then use symmetric encryption to encrypt the data and send it.

From the above analysis, we give the following guidelines for the choice of game protocol type.

1) Weakly networked games: games such as casual and card games can be directly HTTP protocol If security is important, useHTTPS

2) Real-time, high interactivity requirements: these games generally need to maintain long connections, preferring standard ws protocols (while using binary serialization methods such asProtobuf ), you can use the wss protocol if you consider security. The TCP protocol can also be used for native platforms that provide a socket interface, with symmetric encryption of the data to enhance security.

3) Real-time requirements are extremely high: not only need to maintain a long connection with the server, and latency and network jitter are extremely high requirements (such as FPS, racing games), can use UDP-based implementation of the stream transfer protocol such asQUIC , KCP, etc.

7. Concurrency model for network access layer

To handle concurrent requests from clients, there are four common concurrency models on the server side.

7.1) Process.

Processes were the first concurrency model to be adopted, with processes acting as units of operational resource allocation, scheduling, and having independent runtime spaces. The process concurrency model in which each request is handled by a separate process and processes can only handle one request at a time, the biggest advantage of this model is its simplicity. If the process processing the request blocks due to a system call or the process runs out of time slices, the preemptive process scheduler suspends the execution of the old process and schedules the execution of the new process, a process that involves a large overhead context switch and the disadvantage of the process concurrency model is that it is relatively inefficient. The most typical services that use the process model are Apache.

7.2) Threads.

The thread concurrency model is a refinement of the process model, where threads are subordinate to processes and are a more granular execution scheduling unit for the system. Different requests can be handled by multiple concurrently executing threads within the process, which are automatically scheduled by the operating system kernel. The main advantage of threads over processes is that scheduling context switching has less overhead, but because multiple threads share address space, additional inter-thread mutual exclusion and synchronization mechanisms are required to ensure programmatic correctness. Typical services that use a threaded model are Tomcat.

7.3) IO multiplexing.

Using IO multiplexing mechanisms such as epoll provided by the operating system to monitor read and write events on multiple connections simultaneously, IO multiplexing is also known as an event-driven model where the network program execution logic can be abstracted as an event-driven state machine. IO multiplexing avoids read and write blocking, reduces context switching, and improves CPU utilization and system throughput. But IO multiplexing turns what would otherwise be "synchronous", linear processing logic into an event-driven state machine, with the processing logic spread out over a large number of event callback functions. This asynchronous, non-linear model makes it significantly more difficult to program, such as the common callback hell problem with nodeJs. Typical services that use the IO reuse model are Nginx, Netty.

7.4) Concurrency.

Concurrency, also known as lightweight threading, is a collaborative, non-preemptive model of multitasking concurrency. The concurrent process runs in user space, and when it encounters a block or a specific entry, it actively gives up the CPU by explicitly calling the switch method, and the task scheduler picks another concurrent process to execute.

Concurrent switches simply change the execution function stack and do not involve kernel-user state conversions or context switching, and the overhead is much less than process/thread switches. Although the concept of concurrent programming has been proposed for a long time, it is only in recent years that more and more languages (go, Haskell) have built-in support for concurrent programming that developers are familiar with. Concurrent programming greatly optimizes the programming experience for developers, enabling fast implementation of program logic in a synchronous, sequential programming style, and also having the performance of IO multiplexing asynchronous programming. Typical services that use the concurrent model are openresty (Lua), gevent (Python), golang.

7.5) Summary.

The above summarizes four common concurrency models currently in use, which have significant differences in terms of working principle, operational efficiency, and programming difficulty, and each has applicable scenarios that should be carefully evaluated based on requirements when used in practice. In practice if there are no reusable off-the-shelf network components or historical package baggage we recommend using the concurrency model for developing network access layer services.

Appendix: more highlights of articles for further study

[1] Basic information on network programming. TCP/IP Explained- Chapter 11- UDP: User Datagram Protocol》 《 TCP/IP Explained- Chapter 17- TCP: Transmission Control Protocol》 《 TCP/IP Explained- Chapter 18- TCP connection establishment and termination》 《 TCP/IP Explained- Chapter 21- TCP timeouts and retransmissions》 《 Technology Past: The TCP/IP Protocol That Changed the World (precious pictures, mobile phones, click carefully)》 《In Plain English - Deeper Understanding of the TCP Protocol (above): Theoretical Foundations》 《 Easy to understand - Deeper understanding of TCP protocol (next): RTT, sliding window, congestion handling》 《 Theoretical Classics: The 3 Handshakes and 4 Waves Process of the TCP Protocol Explained》 《 Theory to Practice: Wireshark Packet Capture Analysis of TCP 3 Handshake, 4 Wave Process》 《 Computer Network Communication Protocol Relationship Diagram (Chinese Collector's Edition)》 《 What is the maximum size a packet can be in UDP?》 《 P2P Technology Explained (I): NAT Explained - Detailed Principles, Introduction to P2P》 《 P2P Technology Detail (II): NAT traversal (hole-punching) scheme in P2P detailed explanation》 《 P2P technology detailed explanation (3): P2P technology STUN, TURN, ICE detailed explanation》 《 Easy to understand: quickly understand the principle of NAT penetration in P2P technology》 《 The Untold Story of Network Programming (I): A Brief Look at the Troubleshooting in the TCP Protocol (Previous)》 《 The Untold Story of Network Programming (Part II): A Look at the Troubleshooting in the TCP Protocol (Part II)》 《 Little known network programming (III): why TIME_WAIT, CLOSE_WAIT when closing a TCP connection》 《 The Untold Story of Network Programming (IV): A Deeper Look at Analyzing TCP's Abnormal Closures》 《 Lesser Known Network Programming (V): UDP Connectivity and Load Balancing》 《 The Untold Story of Network Programming (VI): Understanding the UDP protocol in depth and using it well》 《 Lesser Known Network Programming (VII): How to make unreliable UDP reliable?》 《 A Lazy Introduction to Network Programming (I): A Quick Understanding of Network Communication Protocols (Previous)》 《 A Lazy Introduction to Network Programming (Part II): A Quick Understanding of Network Communication Protocols (Part II)》 《 Network programming for lazy people (3): a quick understanding of the TCP protocol one is enough》 《 A Lazy Introduction to Network Programming (IV): A Quick Understanding of the Differences Between TCP and UDP》 《 A Lazy Introduction to Network Programming (5): a quick understanding of why UDP is sometimes said to be more advantageous than TCP》 《 Network Programming for Lazy People (VI): the most generalized primer ever on how hubs, switches, and routers function》 《 A Lazy Beginner's Guide to Network Programming (7): In-depth, comprehensive understanding of the HTTP protocol》 《 A Lazy Beginner's Guide to Network Programming (VIII): Handy Guide to Writing TCP-based Socket Long Connections》 《 Making the Internet Faster: Technical Practice Sharing of the Next Generation QUIC Protocol at Tencent》 《 Summary of optimization tools for short connections in modern mobile networks: request speed, weak network adaptation, security guarantees》 《 Talking about long connections for network programming in iOS》 《 Must Read for Mobile IM Developers (I): Easy to understand the "weakness" and "slowness" of mobile networks》 《 Must Read for Mobile IM Developers (II): The Most Complete Summary of Weak Mobile Network Optimization Methods Ever》 《 IPv6 Technology Explained: Basic Concepts, Current Application Status, Technical Practices (Previous)》 《 IPv6 Technology Explained: Basic Concepts, Current State of Application, Technical Practices (next part)》 《From HTTP/0.9 to HTTP/2: An article on the historical evolution and design thinking of the HTTP protocol》 《 Introduction to brain-dead network programming (I): follow the animation to learn TCP three handshakes and four waves》 《 Introduction to brain-dead network programming (II): what exactly are we reading and writing when we read and write to sockets?》 《 Introduction to brain-dead web programming (3): some must-know and must-know knowledge of the HTTP protocol》 《 Getting Started with Brain-Damaged Web Programming (IV): A Quick Understanding of HTTP/2's Server Push》 《 Understanding the technical challenges of real-time communication using the design of the network access layer of an online game server as an example》 >> More in this category ...... [2] Selection of formats, protocols related to network communications. Protobuf communication protocol in detail: code demo, detailed principles, etc.》 《 A Java code demo based on Protocol Buffer》 《 Full review: is Protobuf performance 5x faster than JSON or not?》 《 Technical issues to face in IM development on mobile (with communication protocol selection)》 《 A brief description of those pitfalls of mobile IM development: architecture design, communication protocols and clients》 《 Theory to practice: a set of typical IM communication protocols designed in detail》 《 Sharing of technical practices such as protocol design of the 58-to-home real-time messaging system》 《 Explaining how to use Google's Protobuf in NodeJS》 >> More in this category ...... [3] Other highlights of real-time audio and video development. Instant Messaging Audio and Video Development (I): Theoretical Overview of Video Codecs》 《 Instant messaging audio and video development (II): Introduction to digital video for video codecs》 《 Instant messaging audio and video development (III): video coding and decoding of coding fundamentals》 《 Instant Messaging Audio and Video Development (IV): Introduction to Predictive Technology for Video Codecs》 《 Instant Messaging Audio and Video Development (V): Understanding the Mainstream Video Coding Technology H.264》 《 Instant Messaging Audio and Video Development (VI): How to get started with audio codecs》 《 Instant Messaging Audio and Video Development (7): Introduction to Audio Fundamentals and Coding Principles》 《 Instant Messaging Audio and Video Development (VIII): Common Real-Time Voice Communication Coding Standards》 《 Instant Messaging Audio and Video Development (IX): Overview of Echo and Echo Cancellation for Real-Time Voice Communication》 《 Instant Messaging Audio and Video Development (X): Echo Cancellation Techniques for Real-Time Voice Communication Explained》 《 Instant Messaging Audio and Video Development (XI): Real-time Voice Communication Packet Loss Compensation Technology Explained》 《 Instant Messaging Audio and Video Development (XII): Exploring Multiplayer Real-Time Audio and Video Chat Architecture》 《 Instant messaging audio and video development (XIII): the characteristics and advantages of real-time video coding H.264》 《 Instant Messaging Audio and Video Development (XIV): Introduction to Real-Time Audio and Video Data Transfer Protocols》 《 Instant Messaging Audio and Video Development (XV): Talking about P2P and Real-Time Audio and Video Applications》 《 Instant Messaging Audio and Video Development (XVI): A few suggestions for real-time audio and video development on mobile》 《 Instant messaging audio and video development (XVII): the former life and death of video coding H.264, VP8》 《 Echo Cancellation Techniques for IM Live Audio and Video Chatting Explained》 《 An overview of several key technical indicators that directly affect user experience in real-time audio and video broadcasting》 《 How to optimize the transmission mechanism to achieve ultra-low latency for real-time audio and video?》 《 First Disclosure: How Racer managed to watch a live stream with millions of viewers in the same room and still turn on in seconds without lagging》 《Android live practice: build a simple live system with your hands》 《 Some Optimization Ideas for Live Video Streaming on TCP Data Transfer Layer in NetEase Cloud Letter》 《 Real-time audio and video chat technology sharing: anti-packet loss codecs for unreliable networks》 《 How P2P Technology Reduces Live Video Streaming Bandwidth by 75 Percent》 《 Interview with WeChat Video Technology Leader: The Evolution of WeChat Live Video Chat Technology》 《 Tencent Audio and Video Lab: Real-time video chat in HD with ultra-low bitrate using AI black technology》 《 WeChat team shares: the technology behind WeChat's billion daily live audio and video chats demystified》 《 The implementation ideas and technical difficulties of the recently popular real-time live quiz system are shared》 《 Benefit post: the most complete real-time audio and video development to use the open source project summary》 《 Seven Niu cloud technology sharing: using QUIC protocol to achieve real-time video live 0 lag!》 《 Reflections and Technical Practices on Ultra-Low Latency Architectures in Real-Time Audio and Video Chat》 《 Understanding Latency Issues in Real-Time Audio and Video Chat One article is enough》 《 Live video streaming client technology inventory: Native, HTML5, WebRTC, WeChat applet》 《 An introductory outline of real-time audio and video technology written for the uninitiated》 《 WeChat Multimedia Team Interview: Learning from Audio and Video Development, Audio and Video Technologies and Challenges for WeChat, etc.》 《 Tencent technology sharing: the story behind WeChat small program audio and video technology》 《 Interview with Liang Junbin from WeChat Multimedia Team: Talking about what I know about audio and video technology》 《 Sina Weibo Technology Sharing: The Road to Optimization of Weibo Short Video Service》 《Technical principles and practical summary of mixing of real-time audio in live video applications》 《 Understanding the technical challenges of real-time communication using the design of the network access layer of an online game server as an example

(This article was simultaneously posted on.http://www.52im.net/thread-1915-1-1.html


Recommended>>
1、Vulnerability Scanning and Recurrence Metasploit Penetration Testing for Kali
2、Binhai New City Building the worlds first precision medicine big data cluster
3、Huantai County big data to help the wisdom of sanitation to create a new model of rural sanitation management
4、OOE the worlds first blockchain for green low carbon economy applications
5、RunDeckv301 released server automation

    已推荐到看一看 和朋友分享想法
    最多200字,当前共 发送

    已发送

    朋友将在看一看看到

    确定
    分享你的想法...
    取消

    分享想法到看一看

    确定
    最多200字,当前共

    发送中

    网络异常,请稍后重试

    微信扫一扫
    关注该公众号