Sifting in Corrade's context is the process of applying various operations to Corrade's data output and has been introduced in order to overcome the 2048 byte limitation of the LSL http_request event handler. What happens is that in LSL, the http_request event handler only receives the first 2048 bytes from any source. The result is that when you send a command to Corrade that will send more data than 2048, the LSL script will only be aware of the first 2048 bytes, after which the data will be truncated.

For instance, by using sifting, and given some previous knowledge of the data returned by Corrade, you can truncate the large data output by applying a regular expression to that data. Corrade will then apply the sift and return only the matched groups of the regular expression - excluding the first base match.

For example, consider the following command that retrieves the wearables currently worn by Corrade:

        llInstantMessage(CORRADE, 
            wasKeyValueEncode(
                [
                    "command", "getwearables",
                    "group", wasURLEscape(GROUP),
                    "password", wasURLEscape(PASSWORD),
                    "callback", wasURLEscape(URL)
                ]
            )
        );

when you issue this command a long list of wearable types by names is returned to the callback URL (URL) as a CSV string in the "data" key. For example, the output sent to the callback URL could be something like this:

command=getwearables&group=My Group&data=Shape,"Coco's Shape - Black",Skin,"Kira's Black Skin",Hair,"Bald Head",Eyes,"Blank Eyes"&success=True

Yet, suppose that you do not really care about all the wearables and that you just want to retrieve what is worn in the Skin slot. In that case, you would use sifting and send the command as:

        llInstantMessage(CORRADE, 
            wasKeyValueEncode(
                [
                    "command", "getwearables",
                    "group", wasURLEscape(GROUP),
                    "password", wasURLEscape(PASSWORD),
                    "sift", wasURLEscape(
                        wasListToCSV(
                            [ 
                                "match", wasURLEscape(
                                    ",Skin,([^,$]+)"
                                )
                            ]
                        )
                    ),
                    "callback", wasURLEscape(URL)
                ]
            )
        );

which will return:

command=getwearables&group=My Group&data="Kira's Black Skin"&success=True

As you can observe, the data key now contains only the value that matches the sift regular expression.

Since most data output from Corrade contains a predictable CSV format of keys by values, sifting can be used mainly for two purposes:

• As a filter to filter the data output from Corrade.
• To overcome the 2048 byte limitation by returning only the data you are interested in.

Introduced in more recent progressive releases, Corrade supports various other actions instead of just regular expression matching. These actions are applied in the order in which they are supplied to Corrade. For instance, suppose that for a given command, Corrade would return a data key:

Shape,"Coco's Shape - Black",Skin,"Kira's Black Skin",Hair,"Bald Head",Eyes,"Blank Eyes"

and that we would issue the command with a sift set to:

        llInstantMessage(CORRADE, 
            wasKeyValueEncode(
                [
                    "command", "getwearables",
                    "group", wasURLEscape(GROUP),
                    "password", wasURLEscape(PASSWORD),
                    "sift", wasListToCSV(
                        [ 
                            "take", 2,
                            "skip", 1
                        ]
                    ),
                    "callback", wasURLEscape(URL)
                ]
            )
        );

in that case, take will make Corrade trim the returned data to two elements:

Shape,"Coco's Shape - Black"

after that, the skip parameter will skip the first item in the CSV list, resulting in:

"Coco's Shape - Black"

which will finally make Corrade send the data value as:

data="Coco's Shape - Black"

Another example could be retrieving the region's top scripts and only being interested in the name of the object. You would use the getregiontop command and, given estate manager rights, retrieve the top scripts:

llInstantMessage(CORRADE,
    wasKeyValueEncode(
        [
            "command", "getregiontop",
            "group", wasURLEscape(GROUP),
            "password", wasURLEscape(PASSWORD),
            "type", "scripts",
            "callback", wasURLEscape(URL)
        ]
    )
);

This will make Corrade return a rather long CSV such as:

0.241777,"Kira Komarov",1ad33407-a792-476d-a5e3-06007c0802bf,"Kira Komarov","<156.7988, 165.482, 3401.612>",0.08267129,"Dincii Resident",13a6f33f-c4cf-4a77-8a71-6d15c04c0a5f,"Dincii Resident","<89.98981, 86.85989, 21.35474>",0.03289175,B321,e2bc38fb-194b-5df6-e2f8-ce624dcd19e9,"Tatiana Lapis","<144.8422, 136.2846, 22.11347>"

Now, let us say that we are really only interested in the name of objects and do not care about the rest. We will thus have to extract:

...,"Kira Komarov",...,...,...,"Dincii Resident",...,...,...,...,B321,...,...,...,...

We do that by skipping the first element (the score) and then selecting each fifth element from the CSV:

...,"Kira Komarov",...,...,...,"Dincii Resident",...,...,...,...,B321,...,...,...,...
  ^      ^                             ^
  |      |                             |
  |      +-1--------2---3---4-------5--+
  |
  + skip

The resulting LSL code would then be:

llInstantMessage(CORRADE,
    wasKeyValueEncode(
        [
            "command", "getregiontop",
            "group", wasURLEscape(GROUP),
            "password", wasURLEscape(PASSWORD),
            "type", "scripts",
            "sift", wasListToCSV(
                [
                    "skip", 1,
                    "each", 5
                ]
            ),
            "callback", wasURLEscape(URL)
        ]
    )
);

and will return:

"Kira Komarov","Dincii Resident",B321

If you use sifting, then Corrade's data output for a command will always be one of the following:

• the command has failed and success is set to False.
• the command has succeeded:
  • the sift has succeeded and the data value is replaced by the result of the sift.
  • the sift has failed and the data value is removed.

Note that Corrade does not set the success key to False in case the sift has failed but instead removes the data key indicating there is no output from the command. No data returned from a command that is supposed to return data where a sift was used is usually an indication that the regular expression failed. In such cases, it is useful to send the command without and with the sift to check whether there is data to be matched in the first place and then to see whether the sift succeeded. We will not go into the syntax of regular expression since you can find a large amount of information about them on the Internet but it is helpful to test your regular expression before deciding on it, for example, by using RegEx Hero or RegEx Pal.

The API for sifts can be found on the API page listed along with all available Corrade commands, notifications and configuration keys. A full list of sifts is available along with examples and notes on the various actions that can be performed.