![]() The first time a Go program allocates a single 24 byte struct, the allocator will ask the OS for 8KB, which it will use to satisfy other requests for 17-24 byte objects. And it requests memory from the OS in even larger chunks, or slabs. Go uses a slab allocator: It only allocates memory in specific sized chunks, or size classes. Second, and less obvious, it reduces overhead from the Go memory allocator. Each system thread takes hundreds of kilobytes of memory. First, it reduces the number of system threads. We also set GOMAXPROCS=1 when starting the binary. If you need more memory than you have, even a theoretically perfect garbage collector will not save you. There’s an exciting Go proposal to make this kind of setup easier and more automatic, but even that is not a panacea. This uses more CPU, but reduces memory usage by making it more likely that any given allocation can re-use freed memory. We made the Go garbage collector more aggressive, by calling debug.SetGCPercent. The internet has plenty of discussion of this kind of optimization, and they’re pretty standard stuff: eliminate per-packet allocations, re-use buffers, break some layers, and so on. So another obvious step is to bust out pprof and optimize memory allocations. The core of the Tailscale VPN is written in Go. Still, with baseline operation taking 70-80% of the memory, we decided to start there. Plus, with many devices, we had to update that data more often. The worst problems were for users with large tailnets: There was just more data. This made it clear that baseline operation of Tailscale consumed 10-12MB, leaving only a few MB for anything else. ![]() We added some logging about our memory usage. The first step is to understand the problem. This blog post is about how we tackled the problem, with a bit of philosophizing and a surprise twist at the end. It was a constant low level drain on our engineering team and our product. And the knowledge that doing more work caused more crashes caused us to leave important improvements out of the iOS app, like http2 and UPnP support. When we use too much memory, iOS snipes our network extension, and your VPN access goes down. That has been a constant pain point for our users-and especially for us. ![]() Normal iOS apps can use 5GB or so of memory before iOS kills them. But with this power comes a memory straightjacket. This lets us run in the background, so we can secure traffic from all of your applications, without them having to change anything. ![]() Tailscale on iOS runs as a special kind of app, a Network Extension. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |