12 October 2013
libomv
for SSB compatibility.6 March 2013
17 June 2012
Types.cs
- using AssetType
now.11 June 2012
Rake is a proof-of-concept texture, sculpt, animation, notecard and script backup system that traverses the entire inventory and downloads all the textures it finds. This has been done before by Second Inventory which tops at a cost of $29. Compared to Second Inventory, Rake only backs-up assets instead of entire objects. Another significant difference is that Rake does not provide a graphical user-interface with buttons.
Another interesting feature is that Rake not only downloads the assets but it also classifies them in separate directories. People with tidy inventories will be very happy and one can use free tools such as IrfanView to browse your collections of textures. The design choice was to keep textures as JP2
s instead of converting them to TGA
s in order to stay as close as possible to the real thing.
You will not see thumbnail previews, unless you are using some software like ACDSee or Irfanview which can read JP2
s but PhotoShop (and most photo / texture editing programs) will have no problem with reading the JP2
s.
Rake operates on a one-off basis, it is not meant to run permanently, but rather to be used once to download the entire inventory.
The code relies on the OpenMetaverse Library.
Rake uses a recursive breadth-first algorithm to look for assets. Except the usual log-in and grid-selection, the rake-meter at the bottom is an indication of the current operations taking place. The key is a simple indication of what is currently being downloaded:
Symbol | Description |
---|---|
- | Directory |
T | Texture |
A | Animation |
N | Notecard |
S | Script |
! | Failed to download asset. |
Rake.exe
. Double-click that file to start Rake.1
.
For *nix systems you will have to download the mono compiler (not to be confused with the LSL mono) and install it depending on your distribution. Then you can follow the instructions above with the difference that you will have to run Rake.exe
from the command line using:
mono Rake.exe
JPEG2000
format. These can be viewed (and edited) on Windows using any of the programs capable to read JPEG2000. JPEG2000
images are encoded using CYMK
and in order to convert them to an RGB
color format, you may sometimes have to invert the colors in any application capable of reading JPEG2000
images..animatn
format which is a binary animation format. Most V1.x viewers have the functionality built-in but it has to be enabled using a patch (see Importing Animations bellow). Regrettably this requires the recompilation of the viewer. We are unsure what the exact file format is and whether other animation-editing programs are able to read the .animatn
file format.RTF
, however that is not exactly a rich-text format but rather a Linden developed rich-text-like format. The RTF
is there to instruct the user that they are not dealing with a plain ASCII file. In order to open up the notecards, simply use Notepad
or Wordpad
under Windows which should translate some of the Linden rich text formatting.If you wish the compile Rake yourself rather than downloading the binary, the following steps should provide a reference.
Main.cs
s file and replace it with the contents below.OpenMetaverse.Types
, OpenMetaverse.Utilities
and OpenMetaverse
should be sufficient).You can compile Rake on the command line provided you have the mono compiler installed and you have downloaded the libomv assemblies.
dmcs Rake.cs -r:libomv/OpenMetaverse.dll -r:libomv/OpenMetaverseTypes.dll -r:libomv/OpenMetaverse.Utilities.dll
dmcs
is the mono
compiler for the .NET Framework 4.0
.
C# is an extremely powerful languages that allows lambda expressions and LINC searches. Initially we thought about adding a graphical interface, which would have been easy by using GTK or even Windows forms. However, we thought that it would be redundant since the whole system is designed to be a one shot take.
For example we use the following one-liner to display a line of dashes that goes goes from one end of the terminal to the other:
new List<int>(new int[Console.WindowWidth]).ForEach(i => Console.Write("-"));
which creates a blank list of integers allocated with the length of the window width and then for each non-existing element prints a dash.
The code consists of just one C# file.
/////////////////////////////////////////////////////////////////////////// // Copyright (C) Wizardry and Steamworks 2011 - License: GNU GPLv3 // // Please see: http://www.gnu.org/licenses/gpl.html for legal details, // // rights of fair usage, the disclaimer and warranty conditions. // /////////////////////////////////////////////////////////////////////////// using System; using System.Globalization; using System.IO; using System.Threading; using System.Collections.Generic; using OpenMetaverse; using OpenMetaverse.Messages.Linden; using System.Linq; namespace Rake { internal class Rake { private static readonly GridClient Client = new GridClient(); private static string _firstName = string.Empty; private static string _lastName = string.Empty; private static string _password = string.Empty; private static readonly Thread RakeThread = new Thread(ProcessInventory); private static readonly List<bool> StabilizeChecks = new List<bool>(); private static AutoResetEvent _folders = new AutoResetEvent(false); private static readonly KeyValuePair<char, string>[] Grids = new[] { new KeyValuePair<char, string>('1', "https://login.agni.lindenlab.com/cgi-bin/login.cgi") , new KeyValuePair<char, string>('2', "http://os.bio-se.info:9000/") , new KeyValuePair<char, string>('3', "Manual override.") }; private static bool _overwriteExisting = true; private static readonly List<string> Logo = new List<string> { " ,v+s. ,.\n", " ,z~ Vs ,gW`Vi\n", " _Y` VWs __gmW@@@ ].\n", " g/\\g=YM` V@WmW@@@@@@@@@@@@@@@@@@@@@@@@@ [\n", " Z+/~ _d 7e. Y@@@@@~~~~~~V***@@@@@@@@@@@@@b b\n", " _m@@@Wv.Vm. @@@@@ 'VM@@@@@@@@@@Wm*\n", " gW@@@*fT@i '@ Y@@@@ Y@@@@@@@@@Wm_\n", " ,g@@@*~ ,/. ,/` d@@@@ ]@@@@W 'VM@@@Ws\n", " ,_m@@@*` /Z gf ,@@@@A*f~~~~~+=s_. g@@@@@P '*@f'Ms\n", " K~~******==*f~M[\f` ,g@Af~ ,mms. Y@@@@@Af 'M/\\4W.\n", " !z i/ '+m@@*~` ,gW@@@@@s. i@@@@@| Vbv*W.\n", " gW| M. -' _g=~~M@@@@@@@@@@@@@@@@@m_ 'N-Z@.\n", " i@@W_-. '*s. ,_mK` MK` ]@@@@@@@@*fVN2~ 'M\\_W\n", " ,@@@f'*eK__s__2_gmm@*Y!Vtms ]5__W@@@A*M@@. 'Vc. 'bgVb\n", " W@@P [,._z\\z 'Vmz` M@@@A` ]@@[ '+_ -gs@i\n", " i@Af '8M` K ,_=Y=s M@P @@b '\\s M~VW\n", " Wf ,-'`|~`-. gf,vf~\\s_/ V '* @@@b Vs ][~@i\n", "i` .' . i -'c ,P,_ ' '- !i @@@@._Y~Ve_. 'N. b ~[\n", "! ] i~-L-'. '. ![~V ! b vf**MK. V@D__ Ms @M@[\n", "s ] !,-7-,` ,v=e b i P ,/ ~N. '+ '` ]@b ]_,b\n", "@. `,-` ! - P A !i / [ K . , Y. ,W@@i W-i[\n", "]W_ '=,_L .-`t !s Ys v i[ 'VMf ,g@mm@m____gW@@@@[ A 4[\n", "]@*+=g[ 'i,g=Yc 4@['c _- . ,@[ Yms_d@@@@@@@@@@@@@@@[ ,[ @[\n", " @_ ~- '` ,W[ ]W_v` v g@@ ]@@@@@@@@@@@@@@@@@@A d g@`\n", " ]@@D- __gmmm,_L@@` !A~ ,-' g@@@! ]@@~M@@@@@@@@@@@@Af i[-WP\n", " 5_mm@~ i!]@@@A ds [,gW@@@A` ]@@ 'M@@@@@@**f~ ,AsdA\n", " M@@@s ] ]@@A` g@@@W- `]@@@A~ ]@@ 'M@@@@@ ,P-gW`\n", " M@@@b !['*f g@@Af`_- ,f~ ]@@. 'M@@@@W. ,g@WmA`\n", " V@@@W. 'e__Z~!@K_mmz=f` Wizardry ]@@b '@@@@@@Wsg@@@@f\n", " 'M@@@W. iA '`i and ]@@@i Y@@@@@@@@@@@b.\n", " 'M@@@Ws___zP/ Vs Steamworks ]@@@@s '@@@@@@@@@@@@@Ws.\n", " 'V@@@@A` 'N_ ]@@@@@@s_@@@@@@@@@@@@@@@A!\n", " ~*Mb. _ 'V=e_. ]@@@@@@@@@@@@@@@@@@*~`\n", " '~*Wmmm_______.____ !@@@@@@@@@@@@*f~~`\n", " ~V**@@@@@@@@@@Wmgmmm@@W@@@@@A~~~~`\n", " '~~~********M@@********\n", " \n", " v.\"Dancing hogs in managed states.\" \n\n" }; public static void Main() { foreach (var row in Logo) { new List<int>(new int[(Console.WindowWidth - 70) / 2]).ForEach(i => Console.Write(" ")); Console.Write(row); } Settings.LOG_LEVEL = Helpers.LogLevel.None; Client.Settings.STORE_LAND_PATCHES = true; Client.Settings.ALWAYS_DECODE_OBJECTS = true; Client.Settings.ALWAYS_REQUEST_OBJECTS = true; Client.Settings.SEND_AGENT_UPDATES = true; Client.Settings.USE_ASSET_CACHE = false; Client.Settings.FETCH_MISSING_INVENTORY = true; Client.Network.LoginProgress += HandleLoginProgress; Client.Appearance.AppearanceSet += HandleAppearanceSet; Client.Network.SimConnected += HandleSimulatorConnected; Client.Network.EventQueueRunning += HandleEventQueueRunning; Console.Write("[Rake] : First Name : "); var line = Console.ReadLine(); if (line != null) _firstName = line.Trim(); Console.Write("[Rake] : Last Name : "); line = Console.ReadLine(); if (line != null) _lastName = line.Trim(); Console.Write("[Rake] : Password : "); var info = Console.ReadKey(true); var rnd = new Random(); while (info.Key != ConsoleKey.Enter) { if (info.Key != ConsoleKey.Backspace) { new List<int>(new int[rnd.Next(1, 3)]).ForEach(i => Console.Write("*")); _password += info.KeyChar; } else if (!string.IsNullOrEmpty(_password)) { _password = _password.Substring(0, _password.Length - 1); var pos = Console.CursorLeft; Console.SetCursorPosition(pos - 1, Console.CursorTop); Console.Write(" "); Console.SetCursorPosition(pos - 1, Console.CursorTop); } info = Console.ReadKey(true); } Console.WriteLine(); invalid_overwrite: Console.Write("[Rake] : Overwrite files? [y/n] : "); info = Console.ReadKey(true); Console.WriteLine(info.KeyChar); switch (info.KeyChar) { case 'y': case 'Y': _overwriteExisting = true; break; case 'n': case 'N': _overwriteExisting = false; break; default: goto invalid_overwrite; } invalid_grid: foreach (var grid in Grids) { Console.WriteLine("{0}. {1}", grid.Key, grid.Value); } Console.Write("[Rake] : Select grid: "); info = Console.ReadKey(true); if (Grids.Count(g => g.Key.Equals(info.KeyChar)) == 0) { Console.WriteLine(); goto invalid_grid; } switch (info.KeyChar) { case '3': Console.WriteLine(); Console.Write("[Rake] : Please type a login URI : "); line = Console.ReadLine(); if (line != null) Client.Settings.LOGIN_SERVER = line.Trim(); break; default: Client.Settings.LOGIN_SERVER = Grids.First(g => g.Key.Equals(info.KeyChar)).Value; break; } Console.Write(info.KeyChar); Console.WriteLine(); new List<int>(new int[Console.WindowWidth]).ForEach(i => Console.Write("-")); var login = new LoginParams(Client, _firstName, _lastName, _password, "[WaS] Rake", "1.0"); Console.Write("[Rake] : Starting login..."); Client.Network.BeginLogin(login); RakeThread.Start(); } private static void HandleEventQueueRunning(object sender, EventQueueRunningEventArgs e) { Client.Network.EventQueueRunning -= HandleEventQueueRunning; Console.Write("\n[Rake] : Event queue started..."); StabilizeChecks.Add(true); } private static void HandleSimulatorConnected(object sender, SimConnectedEventArgs e) { Client.Network.SimConnected -= HandleSimulatorConnected; Console.Write("\n[Rake] : Simulator connected..."); StabilizeChecks.Add(true); } private static void HandleAppearanceSet(object sender, AppearanceSetEventArgs e) { if (!e.Success) return; Client.Appearance.AppearanceSet -= HandleAppearanceSet; Console.Write("\n[Rake] : Appearance set..."); StabilizeChecks.Add(true); } private static void HandleLoginProgress(object sender, LoginProgressEventArgs e) { if (e.Status == LoginStatus.Success) { Console.Write("\n[Rake] : Login ok..."); StabilizeChecks.Add(true); } switch (e.Status) { case LoginStatus.Failed: Console.Write("\n[Rake] : Failed Login...\n"); break; } } private static void ProcessInventory() { Console.Write("\n[Rake] : Stabilizing, please wait..."); while (StabilizeChecks.Count < 4) { switch (Client.Network.LoginStatusCode) { case LoginStatus.Failed: return; } Thread.Sleep(1000); Console.Write("."); } Console.Write("\n[Rake] : Raking: "); RakeInventory(Client.Inventory.Store.RootFolder); Console.WriteLine(); new List<int>(new int[Console.WindowWidth]).ForEach(i => Console.Write("-")); Console.WriteLine("[Rake] : All operations completed."); Client.Network.Logout(); } private static void RakeInventory(InventoryBase root) { _folders = new AutoResetEvent(false); Client.Inventory.FolderUpdated += delegate(object sender, FolderUpdatedEventArgs e) { if (e.Success) _folders.Set(); }; _folders.Reset(); Client.Inventory.RequestFolderContents(root.UUID, Client.Self.AgentID, true, true, InventorySortOrder.ByName); _folders.WaitOne(60000, false); var inventoryBases = Client.Inventory.FolderContents(root.UUID, Client.Self.AgentID, true, true, InventorySortOrder.ByName, 60000); if (inventoryBases == null) return; foreach (var ib in inventoryBases) { Thread.Sleep(Client.Network.CurrentSim.Stats.LastLag + (int)Math.Round(new ViewerStatsMessage().AgentPing)); var done = new AutoResetEvent(false); if (ib is InventoryFolder) { var folder = ib as InventoryFolder; Console.Write("-"); RakeInventory(folder); continue; } if (!(ib is InventoryItem)) continue; var item = ib as InventoryItem; switch (item.AssetType) { case AssetType.Bodypart: break; case AssetType.Texture: #region Texture Rake var imageTexture = ib as InventoryTexture; var imageSnapshot = ib as InventorySnapshot; string textureFileName; UUID textureAssetUUID; if (imageTexture != null) { textureFileName = Path.GetInvalidFileNameChars().Aggregate(imageTexture.Name + ".jp2", (current, c) => current.Replace(c.ToString(CultureInfo.InvariantCulture), string.Empty)); textureAssetUUID = imageTexture.AssetUUID; goto download_texture; } if (imageSnapshot != null) { textureFileName = Path.GetInvalidFileNameChars().Aggregate(imageSnapshot.Name + ".jp2", (current, c) => current.Replace(c.ToString(CultureInfo.InvariantCulture), string.Empty)); textureAssetUUID = imageSnapshot.AssetUUID; goto download_texture; } break; download_texture: if (!_overwriteExisting && FindFile(textureFileName, Path.Combine(_firstName + " " + _lastName, "Assets", "Images"))) { Console.Write("."); break; } Client.Assets.RequestImage(textureAssetUUID, ImageType.Normal, (state, asset) => { if (state != TextureRequestState.Finished) return; var texturePath = Path.Combine(_firstName + " " + _lastName, "Assets", "Images", Path.GetInvalidFileNameChars().Aggregate(root.Name, (current, c) => current.Replace(c.ToString(CultureInfo.InvariantCulture), string.Empty))); if (!Directory.Exists(texturePath)) Directory.CreateDirectory(texturePath); File.WriteAllBytes(Path.Combine(texturePath, textureFileName), asset.AssetData); Console.Write("T"); done.Set(); }, false); done.WaitOne(60000, false); #endregion Texture Rake break; case AssetType.Notecard: #region Notecard Rake var notecard = (InventoryNotecard)ib; var notecardNormal = Path.GetInvalidFileNameChars().Aggregate(notecard.Name + ".rtf", (current, c) => current.Replace(c.ToString(CultureInfo.InvariantCulture), string.Empty)); if (!_overwriteExisting && FindFile(notecardNormal, Path.Combine(_firstName + " " + _lastName, "Assets", "Notecards"))) { Console.Write("."); break; } Client.Assets.RequestInventoryAsset(notecard.AssetUUID, notecard.UUID, UUID.Zero, Client.Self.AgentID, AssetType.Notecard, true, (transfer, asset) => { if (!transfer.Success) { Console.Write("!"); done.Set(); return; } var notecardPath = Path.Combine(_firstName + " " + _lastName, "Assets", "Notecards", Path.GetInvalidFileNameChars().Aggregate(root.Name, (current, c) => current.Replace(c.ToString(CultureInfo.InvariantCulture), string.Empty))); if (!Directory.Exists(notecardPath)) Directory.CreateDirectory(notecardPath); File.WriteAllBytes(Path.Combine(notecardPath, notecardNormal), asset.AssetData); Console.Write("N"); done.Set(); }); done.WaitOne(60000, false); #endregion Notecard Rake break; case AssetType.Animation: #region Animation Rake var animation = (InventoryAnimation)ib; var animationNormal = Path.GetInvalidFileNameChars().Aggregate(animation.Name + ".animatn", (current, c) => current.Replace(c.ToString(CultureInfo.InvariantCulture), string.Empty)); if (!_overwriteExisting && FindFile(animationNormal, Path.Combine(_firstName + " " + _lastName, "Assets", "Animations"))) { Console.Write("."); break; } Client.Assets.RequestInventoryAsset(animation.AssetUUID, animation.UUID, UUID.Zero, Client.Self.AgentID, AssetType.Animation, true, (transfer, asset) => { if (!transfer.Success) { Console.Write("!"); done.Set(); return; } var animationPath = Path.Combine(_firstName + " " + _lastName, "Assets", "Animations", Path.GetInvalidFileNameChars().Aggregate(root.Name, (current, c) => current.Replace(c.ToString(CultureInfo.InvariantCulture), string.Empty))); if (!Directory.Exists(animationPath)) Directory.CreateDirectory(animationPath); File.WriteAllBytes(Path.Combine(animationPath, animationNormal), asset.AssetData); Console.Write("A"); done.Set(); }); done.WaitOne(60000, false); #endregion Animation Rake break; case AssetType.LSLText: #region Script Rake var script = (InventoryLSL)ib; if (script.Permissions.OwnerMask != PermissionMask.All) break; var scriptNormal = Path.GetInvalidFileNameChars().Aggregate(script.Name + ".lsl", (current, c) => current.Replace(c.ToString(CultureInfo.InvariantCulture), string.Empty)); if (!_overwriteExisting && FindFile(scriptNormal, Path.Combine(_firstName + " " + _lastName, "Assets", "Scripts"))) { Console.Write("."); break; } Client.Assets.RequestInventoryAsset(script.AssetUUID, script.UUID, UUID.Zero, Client.Self.AgentID, AssetType.LSLText, true, (transfer, asset) => { if (!transfer.Success) { Console.Write("!"); done.Set(); return; } var scriptPath = Path.Combine(_firstName + " " + _lastName, "Assets", "Scripts", Path.GetInvalidFileNameChars().Aggregate(root.Name, (current, c) => current.Replace(c.ToString(CultureInfo.InvariantCulture), string.Empty))); if (!Directory.Exists(scriptPath)) Directory.CreateDirectory(scriptPath); File.WriteAllBytes(Path.Combine(scriptPath, scriptNormal), asset.AssetData); Console.Write("S"); done.Set(); }); done.WaitOne(60000, false); #endregion Script Rake break; } } } private static bool FindFile(string file, string directory) { try { foreach (var d in Directory.GetDirectories(directory)) { if (Directory.GetFiles(d).Any(f => f.Equals(file))) { return true; } FindFile(file, d); } } catch (Exception) { return false; } return false; } } }
Animations are exported with the .animatn
suffix, the file-type being supported by the V1.x series of viewers. Thus, animations have to first be converted using a tool such as Anim2BVH and the resulting BVH file can be imported by the viewer.
Animations are exported with the .animatn
suffix, the file-type being supported by the V1.x series of viewers. Although the V1.x series of viewers support uploading .animatn
files, the file-picker itself does not support .animatn
suffixed animations.
One solution is to patch the V1.x series viewer (Singularity was tested) so that .animatn
files can be uploaded. The following patch is an excerpt from the singularity page:
diff --git a/indra.orig/newview/llviewermenufile.cpp b/indra/newview/llviewermenufile.cpp index 7c449c2..b980116 100644 --- a/indra.orig/newview/llviewermenufile.cpp +++ b/indra/newview/llviewermenufile.cpp @@ -889,7 +889,7 @@ void upload_new_resource(const std::string& src_filename, std::string name, { // Unknown extension // *TODO: Translate? - error_message = llformat("Unknown file extension .%s\nExpected .wav, .tga, .bmp, .jpg, .jpeg, or .bvh", exten.c_str()); + error_message = llformat("Unknown file extension .%s\nExpected .wav, .tga, .bmp, .jpg, .jpeg, .bvh or .animatn", exten.c_str()); error = TRUE;; } diff --git a/indra.orig/newview/statemachine/aifilepicker.h b/indra/newview/statemachine/aifilepicker.h index c862231..16708cc 100644 --- a/indra.orig/newview/statemachine/aifilepicker.h +++ b/indra/newview/statemachine/aifilepicker.h @@ -43,6 +43,7 @@ enum ELoadFilter FFLOAD_WAV, FFLOAD_IMAGE, FFLOAD_ANIM, + FFLOAD_ANIMATN, FFLOAD_XML, FFLOAD_SLOBJECT, FFLOAD_RAW, --- a/indra.orig/plugins/filepicker/llfilepicker.cpp +++ b/indra/plugins/filepicker/llfilepicker.cpp @@ -49,7 +49,7 @@ LLFilePicker LLFilePicker::sInstance; #define AO_FILTER L"Animation Override (*.ao)\0*.ao\0" #define BLACKLIST_FILTER L"Asset Blacklist (*.blacklist)\0*.blacklist\0" // </edit> -#define ANIM_FILTER L"Animations (*.bvh)\0*.bvh\0" +#define ANIM_FILTER L"Animations (*.bvh; *.anim; *.animatn)\0*.bvh\0;*.anim\0;*.animatn\0" #ifdef _CORY_TESTING #define GEOMETRY_FILTER L"SL Geometry (*.slg)\0*.slg\0" #endif @@ -752,8 +752,10 @@ Boolean LLFilePickerBase::navOpenFilterProc(AEDesc *theItem, void *info, void *c } else if (filter == FFLOAD_ANIM) { - if (fileInfo.filetype != 'BVH ' && - (fileInfo.extension && (CFStringCompare(fileInfo.extension, CFSTR("bvh"), kCFCompareCaseInsensitive) != kCFCompareEqualTo)) + if (fileInfo.filetype != 'BVH ' && fileInfo.filetype != 'ANIM ' && fileInfo.filetype != 'ANIMATN ' && + (fileInfo.extension && (CFStringCompare(fileInfo.extension, CFSTR("bvh"), kCFCompareCaseInsensitive) != kCFCompareEqualTo && + CFStringCompare(fileInfo.extension, CFSTR("anim"), kCFCompareCaseInsensitive) != kCFCompareEqualTo && + CFStringCompare(fileInfo.extension, CFSTR("animatn"), kCFCompareCaseInsensitive) != kCFCompareEqualTo)) ) { result = false;