A collection of notes on performance tweaks for Corrade.

It is a good idea (as well as a security precaution) to turn off all the notifications that you do not need. Some notifications are fired very often which will incur a penalty in both RAM and CPU. Corrade is very conservative about resource consumption: when you turn something off, it is really off downright to the managing threads being shut-down and not even running.

A good example, is the membership notification which is configurable for each group:

        <Notification>membership</Notification>

This feature makes Corrade send data about people joining and leaving groups. Second Life does not have this feature and without Corrade you cannot know whether somebody has joined or left a group. In order to be able to tell, Corrade periodically scans (by default every 60 seconds) all the members and compares the list to the list of the previous scan. This makes Corrade wake up the CPU and it does some rather intensive operations - in essence, every 60s, the CPU(s) will spike to scan the groups, gather the new lists, compare to the old lists and then discard the old lists. So, if you do not care about this feature, it is wise to turn it off and Corrade will not even launch the thread.

Something similar happens with the effect notification that tracks viewer effects:

        <Notification>effect</Notification>

again, this is feature that must be implemented by Corrade since Second Life cannot tell when an effect expires - it just does, without notice. In consequence, Corrade must track them all, and for every effect, it must keep a timer in order to be able to tell when the effect expires. Of course, this incurs an additional penalty and, if you do not care about this feature, you can turn it off and then Corrade does not even launch the processing thread.

As a general rule, if you do not have a script or set-up that needs a notification, then turn it off. Later, if you change your mind, or require an additional feature, you can always turn the feature on.

Corrade makes the extra effort to use both names and UUIDs wherever possible. However, working with UUIDs can be much more efficient than working with names. It is a good idea to attempt to use UUIDs wherever you can when sending commands to Corrade. A good example are the commands that take the name or UUID of an in-world item. In case you supply a name, Corrade will have to inspect all the objects to find one that matches the name you passed. However, if you pass an UUID then the object will be found much faster since Corrade does not have to query each object in range.

If you are referring to an in-world item, it is a great idea to use llSensor to find an object and pass the UUID instead of the name to Corrade. Generally, if possible, use Corrade only when you need to execute a viewer-only function but solve as many issues as you can in LSL.

Every time you receive a callback from Corrade (for example, via http_request in LSL), make sure that you confirm receiving it with llHTTPResponse:

    http_request(key id, string method, string body) {
        llHTTPResponse(id, 200, "OK"); // <--- like that
        // ... now do what you want to do ...
    }

if you fail to acknowledge receiving the callback, Corrade will not know about that and wait until a timeout has elapsed.

The number of inventory items on the avatar that Corrade connects to can slow down the startup procedures of Corrade. This is due to the fact that many commands dealing with the inventory first need the inventory to be fetched in the first place. Once the inventory is fetched once, and cached, the next startup should be considerably faster. Nevertheless, it is a good idea to either roll a new avatar for your bot or reduce the inventory to a manageable size.

Some commands, especially those dealing with data-related output, can return a large slice of data and, if Corrade is not running on the same machine as your external script, you may want to compress that output such that less data is transferred between Corrade and your script. You can enable this feature by configuring a compression method in Corrade's configuration file (see the HTTP compression tutorial) and then adapting your external script to handle compressed output. When using compression, there is a slight increase in CPU consumption on both the machine that your external script runs on and on the machine that Corrade runs on. However, by using compression, you will reduce Corrade's traffic a lot over time.

The Internet connection you use can have drastic consequences on Corrade. Second Life makes use of both UDP and TCP - the latter is a bother since TCP contains way more overhead than UDP packets and thus packets require more processing as well as being able to carry less data than UDP packets. In either case, most of the services in SecondLife are an allotment of web-services stitched together. For example, you "request" your inventory from the server (newly over TCP) or, when you look for an avatar, you are pretty much using a search-engine that is TCP based (just like Google). If you run a business in Second Life that makes extensive use of Corrade, it is wise to look for some hosting as close as possible to Linden Lab servers.

In case you are running Corrade under Windows, you may want to consider changing the advanced system settings to adjust the performance to Background services rather than programs. This makes quite the difference and is great if Corrade runs on a dedicated machine as a service.

Second Life uses both UDP and TCP - more recently, a lot of services have been moved through CAPS. It seems that some proxies interfere with CAPS and that Corrade is unable to retrieve the result. For example, trying to access SL through Tor is a funny experience.

Here is a list of common symptoms when you have an upstream proxy that does not play well with SL:

• The inventory does not seem to load entirely; the inventory finally loads but sending inventory-related commands that refer to an item you have double-checked exists returns an error message indicating that the item does not exist.
• Corrade does not seem to know in which groups it is.
• Some notifications such as group messages fail intermitently.

all these symptoms indicate that you are pushing Corrade's packets through an upstream proxy. If you have a complex set-up, first try to disable all proxies - you know what to do. Since Corrade acts as a console application, on Unix systems, there usually is no proxy set for the command-line.

In order to disable all proxies, up to Windows 7, you can open Internet Explorer→Settings→Connections (Tab)→Lan Settings… and untick all the boxes:

Under Windows 10 you can find the setting in Network & Internet:

Just like any viewer, Corrade has to keep track of primitives and avatars in range in order to deliver data speedily and, as such, the draw distance is important to Corrade. In case you use your bot to, say, just perform group invites or if you do not pull data from primitives or avatars, then it is highly recommended to set the draw distance as low as you can - even is a good choice, thereby making Corrade blind and saving up on resources.

The patches for the spectre & meltdown vulnerabilities have an inconvenient side-effect of slowing down most Intel (except Atom) and some AMD processors on both Windows and Linux. One of the propsed solutions for spectre & meltdown was to disable certain CPU features (in particular, speculative execution) which may have impacted performance severely. On some Linux machines, the patches have the effect of running mono applications (such as Corrade) at a very high CPU level; Windows, similarly suffers slowdowns.

In case you are hosting Corrade on a single-user machine, then you might consider disabling the spectre & meltdown patches:

• on Linux
• on Windows

If spectre & meltdown patches were a problem, then after disabling the patches, you should see a considerable decrease in CPU usage when running Corrade (~40% gain).

Corrade tracks its own process CPU, RAM, Threads and NET(work) usage using something named internally a "heartbeat" where statistics are polled and reported using timers. The time between heartbeat reports is fixed at intervals whilst the actual polling of CPU, RAM and network usage occurs every .

All measurements (CPU, RAM, Threads and NET) carried out by Corrade are not instant values but rather a moving average where each value is computed as the arithmetic means between two consecutive measurements. A moving average was preferred because occasional "spikes" (or drops) in CPU (or resource) usage are, in our view, irrelevant because they do not represent a good overview of how resources are being used. For instance, an instant measurement of a 100% CPU spike would not allow inferring that a process "uses a lot of CPU".

As an example, in case Corrade consumes CPU in a second cycle and CPU in the next second cycle, the average instant value stored will be which, when truncated to an integer results in an average CPU usage between the second timespan. One could, introduce fractional CPU usage to handle such cases but reporting a CPU usage of does not really convey too much meaning nor does it help get an overview of the resource usage.

The CPU usage, in particular is clamped to the total number of processors - this is due to the fact that Corrade uses a lot of parallelism to process commands and handle notifications such that, unless pinned manually and externally via other tools to a CPU, Corrade will use all available CPUs on a machine. This makes the CPU usage measurement a lot more meaningful because a CPU usage means of the entire number of CPUs. However, the measurement will differ greatly from Linux' tools such as top that report the CPU usage scaled to all processors: top will report a CPU usage in the interval where CPUS is the total amount of CPUs on a machine such that top will end up reporting CPU usage for a process using in fact of the total CPU usage.

Concerning RAM usage, Corrade measures its own internal working set, as per Microsoft's definition, the value "[…] represents the current size of working set memory used by the process […]". That being said, Corrade reports only the RAM that it is actually using but there is no universally acceptable measurement because the measurements would vary between what meaning you want to observe given distinctions between paged RAM, non-paged RAM, reserved RAM, shared RAM, virtual RAM, etc. Given operating system abstractions, RAM is not "just physical RAM".

Here is a practical example of a re-occurring question that can mostly be answered only approximately: "I have a machine with 5GB of RAM, how many Corrade bots can I run?" The answer is always approximate since a lot of Corrade processes will be sharing the same runtime components (ie, libraries or DLLs) such that stating: "1 Corrade bot takes up 50MB of RAM, thusly 2 Corrade bots will take up 100MB of RAM" is incorrect and just an estimate. Furthermore, the error increases linearly such that concluding that you can only run 100 bots with 5GB of RAM (wrongly inferring that ) is an very large underestimate given that all the 100 bots will be using a lot of RAM that will be shared amongst all of the instances.

Corrade additionally reports the number of threads being used - this includes both idle and running threads. Based on this measurement alone, not much can be said: you can have a fast process that uses 100 threads and a slow process that uses just 1 thread. An idle thread does nothing effectively except occupy a very tiny amount of RAM.

The user is not limited to Corrade's heartbeat and any type of measurement can be expressed via external tools, whatever is more meaningful or useful to the application being created: perhaps a user is interested in measuring CPU spikes or perhaps a user may like to measure the virtual memory size including all RAM used by Corrade plus the shared RAM. For instance Amazon AWS charges for the average cumulative CPU usage such that an occasional CPU spike makes no difference to the overall CPU usage pattern such that users may tolerate CPU spikes in favor of a reduced overall average CPU usage.