cool hit counter Wang Xiaobo: Serverless Salon Sharing with Tongcheng Yilong_Intefrankly

Wang Xiaobo: Serverless Salon Sharing with Tongcheng Yilong


Xiaobo Wang: focuses on high concurrent Internet architecture design, distributed e-commerce trading platform design, big data analysis platform design, and high availability system design. With more than ten years of rich experience in technical architecture, technical consulting, and a deep understanding of the importance of technology selection for e-commerce systems.

Tongcheng Yilong is not quite the same as Tencent, we are a company that mainly deals with air tickets. This company actually does quite a lot of things that are relevant to everyone's lives.

Serverless in the same application, in fact, we can see that what Tencent Cloud just shared is Serverless in the cloud. I'm going to talk about why we're doing Serverless for the same trip. It actually comes from the thought that how far is a SQL to a service? Why are we doing Serverless? It's because we grab time. Time is the real cost to us. Back when we were doing this, we were able to have new releases every day. But on the flip side, it doesn't really matter what you post every day, or it doesn't mean you can have any power. What is the real capacity? Can you go live at night for a product's morning requirements? This one is important for e-commerce. So we thought about doing Serverless.

This was done relatively early, in the second half of 2016, and most of the ones that moved to Serverless were moved up in 2017. Why are we so bold to try it? Because in late 2014 and early 2015, containerize all the applications of the same trip. We completed the microservice switch in 2015. What else did we find to do after these two things were done, we found that they were improved for us, but not revolutionary. Let's go back to this, in really diverse application scenarios SQL is something that's very common and services are something that's very common. How far is the distance from SQL to the service application output if it's actually very, very far. All the questions here cause programming to be unhappy, like where is the test environment, where is the development environment, where is the CASH server, where is the load balancing, because these don't make sense to me, I just want to make it my code. This is something to think about, can you make the distance from a SQL to a service even shorter.

How do we shorten this thing? The first thing is the development environment, the test environment, the production environment, the grayscale environment, all kinds of environments. Everyone think back to the various environments and how they were created when there was a failure. When we do high availability or do online checks we tend to keep a tight rein on the production environment, but we think this breaks down at our expense, but does the development test environment pay much attention? No, it's hung up because of its own hang-up. You actually have less demand, which leads to longer times, and subsequently leads to a less competitive market.

There are also frameworks, which come and go. 3,000 developers in the same company can build a whole bunch of frameworks, but such frameworks tend to have a lot of problems and are not uniform. And then next are these environments causing programmers to be unhappy? But to say that as a good programmer you shouldn't know this? Actually, that's right too. Because of course for a full-stack engineer knows these things, but the problem is that we do engineering, and engineering should have its pipeline and division of labor, so this problem becomes a barrier in the pipeline.

Are these the only obstacles? Not. And deployment. Looking at today's go-live from a traditional look, an app release can be six months, a year. Once I read a Japanese guy say they were doing Japanese insurance. It says it's a big project, six months in development and six months online. What did I say about deployment? Said deployment was quick. Then there's O&M, which is the artificial division of technical staff into business developers, who write code, and another category called O&M engineers. Is there any contradiction between them? We've always said that product students and development students are the most conflicted. But on the other hand, have you ever thought about the fact that it's the operations engineers and programmers who are at odds with each other? Why is that? When the first app went live, I believe there was no Ops engineer, because the Ops work must have been done by the engineer himself. But by the time today's division of labor is so large a problem is found, the gulf grows wider and wider, and eventually there is a fight. Can we scrap O&M, this is not to say we don't want him, but let him hide in the background and make him invisible. There are so many serverless, and one of the big ones in there is ops-less, allowing ops to hide behind technology. Another thing is scaling, this thing is also done a lot, of course, there are also high, automatic scaling, but can never avoid when the traffic rushes over will hang a little, so how to do scaling? The simplest is to bury some servers up, I do not run, today on a billion traffic over, I will put two hundred million servers, do not hang it, expand what capacity? But this resource is wasted, and how to use it. This one looks to be cost-efficient related.

Do these things add up to be a motivation for us to go Serverless? Actually, not yet. Another piece, for our service behavior, like our debugging, imagine the misery we brought after microservicing all of our applications back in 2015. In fact, today frankly we do very well with microservices. Why am I saying today that microservices are difficult? Maybe when we have a small number of services, for example, I spoke to a fellow Nifty student and he said they have a lot of servers. How much of it is in the same ECLA? There are close to a thousand apps for just one airfare operations platform. It's still an app. What's the concept of an app? It's a collage of N services. We have thousands. How many independently deployable microservices do thousands of microservices bring? Very scary. And it's iterating on microservices. The debugging that comes with basically scratching your head and providing a bunch of tools to back him up, this is a huge pain. And performance, every programmer is concerned about performance, but is performance something that every programmer should be concerned about? If from a purely technical point of view everyone should be concerned. But being concerned is not the same as being able to fix it. Because not every programmer can get the performance right. So is there a good way to fix it? Like high concurrency this performs better, this high concurrency to where it turns off and makes it fuse. This would free up the programmer to design the business model.

There's also security, which isn't seamless to every programmer, and how to empower those capabilities through a platform where programmers don't have to do other technical things. All of these things add up to the reason we're going to Serverless. From the time we started designing it, to the time we developed it to go live, it was really far. One of the reasons this is so far away is that there is no way to write code in peace, because the core here is the application thread. And at the heart of the application is the model of the business, the line of business. There were too many tigers on the business line that we had to fight off, so we did Serverless. We're doing this platform in late 2015, and when we're done we'll put the same app that can go into it for the app. Why do I say it's accessible? Because we are also completely private from the bottom, to isolation and so on, we do it thinking that to a large extent we are correcting the trouble that microservices are causing now, so a lot of our framework scheduling is encapsulating and updating the microservices framework, and we'll talk about why that is later.

Make programmers code a little happier. Ten years ago this architecture would be used in the same program, say output an application, then output a service, we schedule it, run it, it feels aesthetically pleasing. Why? Two programmers look at it, nothing is needed. Because this is very common in the verticals that are available. Of course such a problem is found to be problematic when growing up.

With the traditional framework model, it causes an explosion of applications that don't even know which is right. Ten years, you wouldn't even dare to go offline if you wanted to, because there is dependency. Bottlenecks in traffic also come out. Always say that performance is bad because your calls are too long and you're too afraid to move them yourself. So the same journey was revamped and all microservices were done in 2015.

Of course microservicing we did, like a quick move to traditional servers to get some of the non-microservices out of the way. What kind of services are called microservices? I think there has never been a good standard. So much so that microservices are still growing up. Why do you say that? I a real life example, we were more aggressive when we started doing our first microservice transformation, so the platform we chose was a transactional one. Of course it was done successfully online, but when did the problem occur? It happened three months later. A new order placement service has appeared in our order placement service. It is then called the "New Order Service" in the traditional programmer's nomenclature. Another month for a new order service, it's called "New New Order Service"? So the architect approached them and said why did they come out with such a name and why can't they merge? Because it's coming online tomorrow and I don't want to pollute the old one. Half-yearly will appear as a new new new new order service, and others put a bracket after it, do not use, have given up. After the emergence of such a service we found that microservices kept growing and its problem exploded, which was that a large number of microservices were emerging and its operation and maintenance would not change. The original O&M approach was simple, it was a greater basis for judging it, and when a large number of microservices emerged after microservices, it was very messy to deploy and operate.

What is the one thing we say over and over again at this time? It's the reconfiguration. Why do programmers like refactoring so much? Why? Because there will always be interruptions while writing code, and there will always be someone to bring you online early. This one we call the programmer dug himself a hole. Keep it to yourself and say it's okay, I have a free slot in three months I'll reconfigure it. But on the flip side, have these really been reconstructed? I'm wondering why we can't just drop the code and make a new one, the old one doesn't want it. But it's an unrealistic problem because the project is too big. So much so that the refactoring cost is very, very high. Every time you refactor, of course the last thing the boss likes to hear is, why are you refactoring again, didn't you just refactor last month? I haven't done my requirements here yet, so how come I have to refactor them again? That's why for a while we said it was forbidden to mention refactoring projects again. Change the name, people, and call it optimization.

In fact, we found that the system will never solve the problem, so it must be solved with technology. How can we fix it? In fact, think back and say, why we do Serverless, in fact, a starting point is very simple, we go to see our operations students, operations students also like more tools, what is the typical tool, a script. Ops writes a lot of scripts, you've seen Ops scripts where the Ops students say wait a minute, I'll refactor this script. The Ops students will say forget it, I'll rewrite it. There are very few bugs in the script, why? The unit conducting the script is very clear. A file name determines exactly what it does. This service always explodes, and R&D hasn't found out why? So the ops student wrote a script that said to reboot at 12:00. So the name of this script is Reboot it. Simple, right? It doesn't need to be refactored.

Why can't we go about our business logic this way? Can a code script be a microservice? By today it's often said Serverless, we call it function as a service. But I'm called Script as a Service, why? When we have only one function left, it is conceivable that both are solved. It's more understandable to simply call it a script. It's fine to use up the script and leave it, and write another one next time. Of course there would be a problem here, saying what kind of things would fit in there? Today is one of the problems that Serverless is not yet so commonly used in the country is that microservers are also like that, what kind of services are microservices? If you pile the whole thing on top, it probably won't work. Because your whole system architecture or transaction class, you're going to find the whole system architecture piece by piece, bit by bit, and it's very expensive to find a thing to string with each other. So not everything is actually suitable for putting. What's appropriate to put? It's just ready to go fit to put in. How many such applications are there really? In fact, a large number of such applications in the Internet, why, it is inherently an adaptive diffuse, highly concurrent industry. Of course for something like a transactional class, like a supply chain type, which is very, very stable and business heavy, cutting to Serverless is not appropriate.

One of the problems I have is that programmers make sure to fill a hole for themselves when they do a new upgrade. So we filled the microservices hole at Serverless

Having said that to sum it up, the business is growing too fast and we need faster code realisation speed to improve performance. There are also fast among, environment issues, deployment issues, application resilient design issues that plague programmers too much, so we need to abstract them out and not have them coupled again. So we set it up so that for the same program we are better off containerizing the application and at the same time making middleware, how are these middleware being used? And of course the use of middleware is actually a key point of change for us at Serverless, why? A question that comes up when we have over 3,000 programmers developing because some of the middleware is made by us and how many programmers don't know how to use it every day, such a question would be asked to the middleware developers. Often middleware is made by five or six people, and this seems to have a bug when used in the face of three thousand programmers. So we also set up a customer service team at that time. This team did it for a month and also went crazy because they were asking the same question every day and had to say it two hundred times a day.

Wouldn't we say make a container our instance deployment and just pull up when we need to be resilient? But that's too much overhead. You deploy with just a little bit of everything and pull up containers that you can't possibly keep up with. So we started going for it. Of course when we did it we transformed the containers, at first we chose Node Js and LUA. Another benefit is that they are dynamic and it is easy for us to retrofit them, we do the isolation and then run them into the container. Wasn't there more to the microservices just mentioned when they became scripted? For this we did an orchestration thing, which is much more complex than microservice orchestration, which is very, very important at this point. But then at this point it says, what's wrong with discounting several scripts strung together? In fact, the intermediate call must be the IP way of the network, all kinds of ways will be there, and this way of tuning must be to the performance degradation. So when it comes to making functions, we choose to do it for lighter applications, and performance is much lower with heavier ones.

This is what we did at the time when we started, and after we were done going live with the first version we did a test where we deployed 4 physical servers with a minimum unit to support up to 10,000 applications. This is of course one of our prototypes, and when we run the prototype, the ones in red are the ones that are heavier instances or have reached a critical value and need to be expanded. The green is ready for expansion, the white is normal. Of course all of our applications will now have such monitoring.

Just said, how to use middleware, this is also a problem to be solved by Serverless, so we made an SDK to unify the use. Why do you do this? As I said earlier there are so many asking questions that you can't service them over, or there is no uniformity in the standards used. This isn't the most important either, we just said we want to do microservices. When there is no server, for example, how do I do it when I need to fish out a batch of data from a drop? First of all, where are the drop servers? The thing itself isn't server enough, you're still programming on the server, so the programmer doesn't want to know where the drops are, or know how many masters and how many slaves it is, I just want to know what its instance name is. What is the benefit of this, when actually developing many students no longer need to think about where to lay their cloth. In the meantime Cash will do it and Q will do it. That way it just keeps saying in the script that something is needed, and that something is always an object.

Doing this also has the problem of scaling, which is also very troublesome, so we redo the entire load layer all over again with a dynamic scaling. Why? Let it perform without degrading performance to get it to pull up quickly. Also we are worried about this hanging when it goes live, and preventing it requires a good disaster tolerance.

We also made SDKs for the back, each of the things we just mentioned. What is more convenient for us is that there are actually things that will do debugging. When debugging, you will find that the logs cannot be exported, on the platform as well, the logs of any case can be exported, the call is from Cash to IDE. It's actually possible that a script also tunes a real microservice, quite normally, and in one case just now, the name of the Serverless service can be found. And control center, load balancing none of that is needed anymore.

The first look at the system when it was made and given to us developers was a happy one, saying that we could write code. The second is that no one uses it at all. I say all that, it's developed, it's used, it's broken the way I was programming. The original programming was debugged from local, and it became fragmented and hard to use when I was still using the set. So we made a set of WEBs with a unified IDE. This is in any case on a microserver, any product can be written, you can write it at home, you can write this code with a bit of time. What are the benefits of it? The original configuration extinguished the environment and there was really no environment left. It's written here without regard to where the environment is, I just need to know my code. Like this script is live and has an access environment that is regionally visible, or something. This has the advantage that there are no environmental constraints in the development process. And based on the range available on the line, it's the data source, the environment, behind the tune that automatically gives him a choice.

Of course the set, Serverless debugging, and we gave it a simple Debug. In fact, it's so well said that the programmer resents it, saying you've scrapped me, I'm leaving you here I can't write code. But it does run into this problem because the programmer can only use yours. So we did a local environment, custom made to do a few things. But the ones that are really best used down the line are the ones that are used more by now, because programmers are under pressure to cash in, in addition to their own technical hobbies. Because this thing will be live in the afternoon when it's done really.

This is still pretty rich, we did code version conflicts and the code went live. In order to say be able to let all programmers put this script, what scripts they have out there, we made a script store, in every department, in every process, there are already these that can be tuned directly.

Another important thing to really do is to say let's have high concurrency in our scripts what do we do, to configure it, we'll have a properties interface. Like many pages that need anti-pickup, and those that need high concurrency, can be used as needed. We even did a spike, an easy spike that we did, which was a queuing system. This time you can imagine something like this, in fact, if a spike requirement, if it is really given to a programmer to do, he may need a week to think about concurrency, but he really needs to finish writing this, probably in an afternoon, and after writing it, it will be ready to use. This spike is not a big one, like last night when there was no room left at 7 or 8pm for a spike, it was all in a flash. Is there any use for what you wrote yesterday? It's not really useful anymore, so just go offline, don't want it, and talk about it later. So with this system you can go live quickly and flexibly.

Each person also has their own directory when they go in, including a request log. Basically each programmer goes in through the portal and sees the logs and their own services.

What did we save when we did it and summed it up for ourselves? We save the cost of project initialization. We're in a project where all sorts of things are omitted from the application, the modules, the framework options. The next thing is the data source, I have to find the logs, I have to go select the data source, etc., and those things are probably reduced by 80%. There's also development, because actually debugging and finding internal dependencies is a pain in the ass during development, and that thing is taken care of in this. Of course the code still has to be written, so it doesn't say 80%, we figure it's 40% efficiency. Of course, for O&M deployment, from the programmer's point of view, O&M involvement is very low; but from the O&M point of view, the involvement is very high because the backend all depends on their ability, so from the O&M point of view it is a 90% improvement, but for the O&M students from the O&M state to the maintenance state.

Let's look at what kind of things we've put on this platform that we've been running for two years. The first one, all of our pages, event promotions, some of the backend with a lot of changes, real time changes are put in. Why do you say the first page of choice? Especially some event promotion pages that change very quickly. Why? Because the market grasp is not very big, so also need to experiment, this experiment will need fast cashing ability, this can be achieved through Serverless. Your deployment, your script deprecation or rewriting this code cost is very low, and with these two prerequisites you will cash in much faster.

Serverless in the same context: light services. Some business services with simple logic, some business services with fast changing logic, lots of small ad hoc services.

Then down comes the calculation of prices in real time. Why is this thing our main scenario as well? In fact, OTA, we do tickets, hotels and such, OTA price calculation and ordinary traditional e-commerce price calculation is not the same. What do people say is different about this? A thousand bucks for a ticket is a thousand bucks. That's not really true. When we buy a hat on XX e-commerce platform, it's 20 in the morning and I believe it's still 20 in the afternoon. But if you buy a ticket or a hotel, the price must be different every time you check. Why is it different? Because the whole planet is sharing a hotel room, or the same seat on the same plane, it's going to get sold to another platform any day now because there are only so many resources. And that's when the price changes with a certain number of people, or the time of the ride. What does that mean? Like yesterday on booking hotels in Beijing, which must have gone up because of the high volume. This one is not OTAs raising prices, but the hotels themselves. This time the OTA makes sure you book a room, so it has to do real-time price calculations. There is a great deal of price logic here. Same with airline tickets, like international tickets, this question like how much does a flight from Beijing to Paris really cost? There are multiple transit. The various transits bring different prices. This must not be allowed to do static or to do Cash pricing. Guaranteed accuracy means real-time forecasting. The problem with real-time forecasting is that there are a lot of price scripts to run. This is the set that is put in our system to do, and this is related to our expansion because this can be discarded when it's done. There's also things like scaling, where having 10 billion visits a day is completely different from 100 million visits, and that elasticity space would be very large.

And of course we're still doing that, like we just recently added more languages, WEB-enabled IDE capabilities were increased, more skills were configured to incorporate automated testing systems. That's all I have to share today, thank you.

Q&A

Q: I'd like to know if our data is on the same trip or on the same server?

A: The same program is a hybrid cloud state because there are more applications and larger volumes. The same trip has a free machine room, which is larger. There are also a large number of virtual machines on the Tencent Cloud. It's used for different things. For example, in many cases traffic expansion relies on Tencent cloud expansion, many supply chain systems, or computing has its own server room, basically a hybrid cloud state.

Q: Not wanting sensitive information to be accessible outside, do you feel comfortable putting it on cloud storage or your own storage?

A: A more challenging one, I don't think where it's placed is the most important question. It's like whether it's safer to put your money in the bank or in your own home. Different people give you different answers. For example, if you add that it's a large estate with thousands of security guards, I reckon it's safer to leave it at home. If you're the only one in your house and it's a rental, it's probably safer to put it in the bank. Of course there are situations where banks are not safe, as they are the biggest targets for robberies. Going back to your earlier question, instead of thinking about where to put your data to be secure, do a good job of securing the overall application. Just said the problem of security, in fact, today's security problems, we search, in the end, how many security problems are caused by the underlying layer, in fact, not much, more is the application of the vulnerability and application of inattention, resulting in the application of the interface of the vulnerability is more, produced by the data insecurity. Instead of going for the bottom layer, we can get the upper layer really good for application security; at the same time we have a security team that will go to the outermost to innermost layer to do application security, host security, vulnerability detection and collection are all done by the security team. In a case like this, it's safety by safety.

Q: We had run into the problem that if we use the cloud to store our information, for the insurance industry, they have people from the insurance commission who check this information every month and need to check this information with them. But every time there is a problem with some data during the reconciliation, there is a problem with the data we want to directly check the database and then make changes directly. If it's on the cloud, we may not be able to access the database directly and modify it if something goes wrong. This is even more problematic if you use Tencent to modify it.

A: You want to do data security, the data must not be modified at will. It's actually also within the same program, under our private cloud, and the application programmers don't have access to the database. This actually does we have a lot of optimization going on, in fact there are tons of corrections to the system to do. The benefit of the system is the security of the data. Because it actually does have a lot of problems occurring for each engineer to go and manipulate the database directly.

Q: We're doing a lot of traditional project development right now, but we come across relational databases. As mutual scaling, mutual models will become more and more correlated, how about this piece of servicelessness to take the traditional project, the complex logic inside the simple split, and make each operation between each service, so that their state is completely unrelated, stateless. I think there's definitely a long chasm in between going from traditional project development, to serviceless, and as you mentioned earlier, how far SQL statements go to our services. So I think this process in the middle is what we're missing the most. So not sure how this process should work out?

A: A traditional application, especially the old application, is a big database to an application situation how to turn it into a Serverless situation, in fact, this has two ways to go: first, with Serverless completely unrelated, need to decouple in the application. That's another topic, probably related to microservices, is how to decouple old applications to service them. If you split directly against this, because you just said that Serverless is not a case of one without the other for microservices, they both should be parallel states, applied to different scenarios. This one decouples the data for the old scenario. After this thing is done, inside pick and choose later out suitable for Serverless, such as more changes, lighter applications, so that the Serverless turn off, which is very, very low pressure for later. So start by decoupling the business model and decoupling the servitization of the application. It's not possible to be a traditional type of architecture model one day and tear it down the next day, that's certainly not realistic because it's impossible to stop and do this. Our original splitting process must be to pick out our more troublesome parts, and the better part to change it to go to the right selection, do version iterations, such as what will happen after a month, what will happen after three months, migrate part of it away, to ensure that the business is still in a concurrent state to go.

Q: Can we eventually get to an ideal state where the whole backend of the project is all serviceless away?

A: This is something that we actually haven't done now after two years of doing it, why? Because what I just said in the PowerPoint, I don't think the current servicelessness is able to completely set up all the systems here. Because it's still doing something in a specific scenario, after all. So there are still plenty of microservices that exist to do its business logic. Why? Just now the whole PPT through down is about concurrent fast and light can move here. But a heavy, unstable one with a microserver might be appropriate.

The PPT attachment to this paper is below.


Recommended>>
1、Baidu is in a mad rush so why does it have to mass produce unmanned cars in 2018 The consequences are worse than you think
2、What is the difference between Oeanc the ocean chain and a regular blockchain
3、The most important thing is that you can use the cheat software to make your own gold flowers
4、When the phone has only 5 charge left
5、5 Black Tech Sites That Teachers Have Treasured For Years Netflix Hate To See It

    已推荐到看一看 和朋友分享想法
    最多200字,当前共 发送

    已发送

    朋友将在看一看看到

    确定
    分享你的想法...
    取消

    分享想法到看一看

    确定
    最多200字,当前共

    发送中

    网络异常,请稍后重试

    微信扫一扫
    关注该公众号