Question

If I set threadsafe: true in my app.yaml file, what are the rules that govern when a new instance will be created to serve a request, versus when a new thread will be created on an existing instance?

If I have an app which performs something computationally intensive on each request, does multi-threading buy me anything? In other words, is an instance a multi-core instance or a single core?

Or, are new threads only spun up when existing threads are waiting on IO?

Answer 1

The following set of rules are currently used to determine if a given instance can accept a new request:

if processing more than N concurrent requests (today N=10): false
elif exceeding the soft memory limit: false
elif exceeding the instance class CPU limit: false
elif warming up: false
else true

The following of total CPU/core limits currently apply to each instance classes:

CLASS 1: 600MHz 1 core
CLASS 2: 1.2GHz 1 core
CLASS 4: 2.4GHz 1 core
CLASS 8: 4.8GHz 2 core

So only a B8 instance can process up to 2 fully CPU bound requests in parallel.

Setting threadsafe: true (Python) or <threadsafe>true</threadsafe> (Java) for instances classes < 8 would not allow more than one CPU bound requests to be processed in parallel on a single instance.

If you are not fully CPU bound or doing I/O, the Python and Java runtime will spawn new threads for handling new request up to 10 concurrent requests with threadsafe: true

Also note that even though the Go runtime is single threaded, it does support concurrent requests:
It will spawn 1 goroutine per requests and yield control between goroutines while they are performing I/O.