« Réunion du 22-02-2022 » : différence entre les versions
Ligne 1 : | Ligne 1 : | ||
source : http://opensimulator.org/wiki/Chat_log_from_the_meeting_on_2022-02-22 | source : http://opensimulator.org/wiki/Chat_log_from_the_meeting_on_2022-02-22 | ||
= Introduction Migration .net5= | = Introduction Migration .net5= | ||
* [https://www.npgsql.org/ npgsql] permet aux programmes écrits en C#, Visual Basic, F# d'accéder au serveur de base de données PostgreSQL . | |||
[11:03] Gavin.Hird @grid.xmir.org:8002 : Salut Andrew | [11:03] Gavin.Hird @grid.xmir.org:8002 : Salut Andrew | ||
Ligne 72 : | Ligne 74 : | ||
[11:11] Ubit Umarov : et d'autres choses encore | [11:11] Ubit Umarov : et d'autres choses encore | ||
[11:11] Vincent.Sylvester @hg.zetaworlds.com:8002: | =Tests avec MariaDB= | ||
[11:12] Ubit Umarov: ms | [11:11] Vincent.Sylvester @hg.zetaworlds.com:8002 : En parlant de librairies alternatives, je me suis posé l'autre jour et j'ai cloné les interfaces [https://fr.wikipedia.org/wiki/MySQL Mysql] et réécrit les migrations pour changer un tas de champs varchar et char en UUID commec'est maintenant supporté par [https://mariadb.org/ MariaDB]. Les tests unitaires passent, bien qu'ils utilisent toujours le connecteur Mysql par défaut et je n'ai pas fait beaucoup de tests pour savoir si toutes les parties fonctionnent. Ubit a cherché un connecteur MariaDB directement, mais il semble que, tout comme npgsql plus récent, il soit livré avec un tas de librairies stupides. J'ai vérifié la longueur des tables et dans l'ensemble des tables, j'ai économisé plus de 280 octets juste en changeant ces champs, ce qui revient à près de 30% de la taille de la structure globale. | ||
[11:12] Ubit Umarov: | |||
[11:12] Ubit Umarov: | [11:12] Ubit Umarov : ms a refusé de supporter l'image sur toutes les plateformes, comme linux. | ||
[11:12] Ubit Umarov: | |||
[11:13] Misterblue Waves: | [11:12] Ubit Umarov : à cause de [https://www.mono-project.com/docs/gui/libgdiplus/ libgdiplus], [https://fr.wikipedia.org/wiki/Cairo_(logiciel) Cairo], etc. | ||
[11:13] Ubit Umarov: | |||
[11:13] Misterblue Waves: | [11:12] Ubit Umarov : ils refusent d'assurer la maintenance | ||
[11:14] Ubit Umarov: maria | |||
[11:14] Andrew Hellershanks: | [11:12] Ubit Umarov : un peu compréhensible | ||
[11:14] Ubit Umarov: | |||
[11:15] Andrew Hellershanks | [11:13] Misterblue Waves : certains problèmes ne devraient pas être volontaires. | ||
[11:13] Ubit Umarov : le Cairo et le [https://fr.wikipedia.org/wiki/Pango Pango]... | |||
[11:13] Misterblue Waves : J'ai utilisé MariaDB comme base de données. Pas de problèmes, mais je ne suis pas un testeur de gros chargements. | |||
[11:14] Ubit Umarov : maria est maintenant ce que les distros aiment. | |||
[11:14] Andrew Hellershanks : C'est le cas depuis un certain temps. | |||
[11:14] Ubit Umarov : je suppose que c'est comme ça depuis qu'Oracle a pris le contrôle de Mysql. | |||
[11:15] Andrew Hellershanks acquiesce. | |||
[11:15] Vincent.Sylvester @hg.zetaworlds.com:8002: 10.7 was released last week or so, now has UUID field type, which is a bit smaller in size to store UUIDs, thing is, to take advantage of that kinda need to find a way to interface with it on a lower level than pushing strings in order to get performance out, just changing the fields to UUID just reduces database size overall, query performance not so much | [11:15] Vincent.Sylvester @hg.zetaworlds.com:8002: 10.7 was released last week or so, now has UUID field type, which is a bit smaller in size to store UUIDs, thing is, to take advantage of that kinda need to find a way to interface with it on a lower level than pushing strings in order to get performance out, just changing the fields to UUID just reduces database size overall, query performance not so much | ||
[11:16] Vincent.Sylvester @hg.zetaworlds.com:8002: Need a direct binary conversion or something to interface with it, but for that you need a connector and the one that is on nuget needs some hacks to work | [11:16] Vincent.Sylvester @hg.zetaworlds.com:8002: Need a direct binary conversion or something to interface with it, but for that you need a connector and the one that is on nuget needs some hacks to work |
Version du 23 février 2022 à 21:55
source : http://opensimulator.org/wiki/Chat_log_from_the_meeting_on_2022-02-22
Introduction Migration .net5
- npgsql permet aux programmes écrits en C#, Visual Basic, F# d'accéder au serveur de base de données PostgreSQL .
[11:03] Gavin.Hird @grid.xmir.org:8002 : Salut Andrew
[11:03] Andrew Hellershanks : Bonjour à tous.
[11:04] Ubit Umarov : Code de gravité Description État de suppression de la ligne de fichier du projet Erreur CS1069 Le nom de type 'PerformanceCounter' n'a pas pu être trouvé dans l'espace de nom 'System.Diagnostics'. Ce type a été transféré à un assembly 'System.Diagnostics.PerformanceCounter, Version=0.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. Pensez à ajouter une référence à cet assembly. OpenSim.Framework.Monitoring D:\Avination\test\opensim\OpenSim\Framework\Monitoring\ServerStatsCollector.cs 65 Actif
[11:04] Ubit Umarov : par exemple
[11:04] Ubit Umarov : les compteurs de performance ont également disparu.
[11:04] Andrew Hellershanks : Je me suis assoupi après le déjeuner jusqu'à ce que mon Assistant numérique personnel me rappelle l'heure par un bip.
[11:04] Ubit Umarov : cela signifie aussi que certains de nos codes de statistiques ont disparu.
[11:05] Ubit Umarov : sur la migration vers .net5
[11:05] Kayaker Magic : J'ai entendu dire que Mike Chase a réussi à compiler OpenSim et à le faire fonctionner avec .net sur des serveurs Windows, mais il a été surpris de constater que le code résultant ne fonctionne pas sur linux.
[11:05] Ubit Umarov : La partie la plus effrayante est l'API qui ressemble à la même chose, mais qui ne l'est pas.
[11:06] Ubit Umarov : eh bien! pour faire cela il devra enlever des choses comme
[11:06] Ubit Umarov : la carte du monde
[11:06] Ubit Umarov : etc...
[11:06] Ubit Umarov : puis réécrire avec des bibliothèques alternatives...
[11:07] Ubit Umarov : les textures lludp sont également mortes.
[11:07] Ubit Umarov : bien comme je l'ai dit, une tonne de choses.
[11:07] Ubit Umarov : je ne suis pas sûr que tu aies vraiment lu qu'il l'avait fait.
[11:07] Ubit Umarov : Je sais qu'il y travaille.
[11:08] Ubit Umarov : de mon côté, aussi comme je l'ai dit, pas de précipitation.
[11:08] Ubit Umarov : il ne faut pas se précipiter.
[11:08] Andrew Hellershanks : C'est un peu bizarre que Microsoft(MS) fasse de tels changements dans .net 5 au point de casser complètement le code du passé.
[11:08] Ubit Umarov : ohh vous devriez voir .netcore 2
[11:08] Ubit Umarov : encore pire
[11:08] Ubit Umarov : même 3
[11:09] Ubit Umarov : c'est à ce moment que MS a vu qu'ils ne pouvaient pas forcer tout le monde à réécrire toutes les applications.
[11:09] Ubit Umarov : donc ils ont ajouté beaucoup plus de .net4.8 dessus.
[11:10] Andrew Hellershanks : Je me demande s'ils ont vu le problème en interne. Cela les a probablement incités à faire quelque chose à ce sujet.
[11:10] Ubit Umarov : mais il y a encore beaucoup de choses qui manquent, comme les images, etc...
[11:10] Ubit Umarov : les parties qu'ils ont ajoutées... pour Windows seulement.
[11:10] Ubit Umarov : net5-0-windows a un code image.
[11:10] Misterblue Waves : je me demande si PrimMesher et libomv peuvent être mis à jour vers .NET6... c'est un projet de week-end.
[11:11] Ubit Umarov : même problème.
[11:11] Ubit Umarov : impossible
[11:11] Ubit Umarov : ça dépend des bitmaps
[11:11] Ubit Umarov : et d'autres choses encore
Tests avec MariaDB
[11:11] Vincent.Sylvester @hg.zetaworlds.com:8002 : En parlant de librairies alternatives, je me suis posé l'autre jour et j'ai cloné les interfaces Mysql et réécrit les migrations pour changer un tas de champs varchar et char en UUID commec'est maintenant supporté par MariaDB. Les tests unitaires passent, bien qu'ils utilisent toujours le connecteur Mysql par défaut et je n'ai pas fait beaucoup de tests pour savoir si toutes les parties fonctionnent. Ubit a cherché un connecteur MariaDB directement, mais il semble que, tout comme npgsql plus récent, il soit livré avec un tas de librairies stupides. J'ai vérifié la longueur des tables et dans l'ensemble des tables, j'ai économisé plus de 280 octets juste en changeant ces champs, ce qui revient à près de 30% de la taille de la structure globale.
[11:12] Ubit Umarov : ms a refusé de supporter l'image sur toutes les plateformes, comme linux.
[11:12] Ubit Umarov : à cause de libgdiplus, Cairo, etc.
[11:12] Ubit Umarov : ils refusent d'assurer la maintenance
[11:12] Ubit Umarov : un peu compréhensible
[11:13] Misterblue Waves : certains problèmes ne devraient pas être volontaires.
[11:13] Ubit Umarov : le Cairo et le Pango...
[11:13] Misterblue Waves : J'ai utilisé MariaDB comme base de données. Pas de problèmes, mais je ne suis pas un testeur de gros chargements.
[11:14] Ubit Umarov : maria est maintenant ce que les distros aiment.
[11:14] Andrew Hellershanks : C'est le cas depuis un certain temps.
[11:14] Ubit Umarov : je suppose que c'est comme ça depuis qu'Oracle a pris le contrôle de Mysql.
[11:15] Andrew Hellershanks acquiesce.
[11:15] Vincent.Sylvester @hg.zetaworlds.com:8002: 10.7 was released last week or so, now has UUID field type, which is a bit smaller in size to store UUIDs, thing is, to take advantage of that kinda need to find a way to interface with it on a lower level than pushing strings in order to get performance out, just changing the fields to UUID just reduces database size overall, query performance not so much [11:16] Vincent.Sylvester @hg.zetaworlds.com:8002: Need a direct binary conversion or something to interface with it, but for that you need a connector and the one that is on nuget needs some hacks to work [11:16] Andrew Hellershanks: If the code uses cairo, pango, etc. they should still be usable in the OS code as external libraries. [11:16] Ubit Umarov: as i told you, that takes a split on the dbs [11:16] Ubit Umarov: and use another connector [11:17] Ubit Umarov: bitmap on mono depends on pango, cairo and libgdiplus [11:17] Ubit Umarov: that ms refuses to use [11:17] Ubit Umarov: so no image code for linux on .net5 or 6 [11:17] Ubit Umarov: as i said .netx-windows does have it.. windows only [11:18] Andrew Hellershanks: Why can't the code just use those libraries anyway even if MS won't support them in .net? [11:18] Ubit Umarov: bc they will not add to .net, code that depends on libs they refuse to use [11:18] Gavin.Hird @grid.xmir.org:8002: no money in that [11:18] Andrew Hellershanks: Other projects use cairo, pango, etc. so those libraries are still around. [11:19] Gavin.Hird @grid.xmir.org:8002: MS has always been about locking in the customers [11:19] Ubit Umarov: and .netx code is also for android and ios [11:19] Andrew Hellershanks: That makes no sense. If you use .NET you have no way to use some other 3rd party library?? [11:20] Ubit Umarov: seems they also think no use really uses it [11:20] Ubit Umarov: seem some telling only seen code to make q-code :) [11:20] Ubit Umarov: they tell to use other image libraries for .net [11:21] Ubit Umarov: fine [11:21] Ubit Umarov: just means all our code will need a rewrite [11:21] Ubit Umarov: jpeg2000 code, scultp maps.. maps.. etc [11:22] Ubit Umarov: dynamic textures [11:22] Vincent.Sylvester @hg.zetaworlds.com:8002: A lot of that stuff is pretty abstract too, figuring out how that currently works is a big task in itself much less rewriting it [11:22] Ubit Umarov: all gone, until a rewrite with something else [11:22] Ubit Umarov: think i only did list concret things [11:22] Misterblue Waves: what our the "supported" target platforms? Window10/11, Ubuntu (mono6), IOS ? [11:23] Andrew Hellershanks: The simpler solution is to write our own code to replace the missing/unsupported bits so the main code doesn't need to be completely rewritten. [11:24] Vincent.Sylvester @hg.zetaworlds.com:8002: Andrew you want to write an image library, sure, I'll schedule the release party, see you in 2035 [11:24] Ubit Umarov: well and still no need to rush [11:24] Ubit Umarov: sadly mono lost a lot of fixes but still around [11:24] Ubit Umarov: so no need to rush [11:25] Ubit Umarov: any change to .net5(6 also means to lose a lot of good hardware outthere [11:25] Andrew Hellershanks: If current/older .NET is using pango, cairo, etc. Why not just make a library that is an interface between the current code and the backend libraries that were already being used? [11:25] Ubit Umarov: just because that hardware can't run last windows [11:25] Ubit Umarov: or even last linux [11:25] Andrew Hellershanks: What does MS say to developers about moving to .NET 5/6? Rewrite your programs? [11:26] Vincent.Sylvester @hg.zetaworlds.com:8002: I'm more worried that latest mono no longer supports postgres without massive changes, not sure if mysql is not to follow soon [11:26] Ubit Umarov: yes that is what they say [11:26] Ubit Umarov: as i said where a lot worse in the times of .netcore 2.0 [11:26] Misterblue Waves: @Andrew it's true that we don't use all the features of Image. Most of the "will not support" stuff is around the functions that reach into Images and twiddle the internal formats (for preformance and stuff) [11:26] Ubit Umarov: bc most code jsut did not run on it [11:26] Andrew Hellershanks: If they keep doing that it is a great advertisement for why you should not use .NET in your programs. [11:27] Ubit Umarov: go tell original opensim devs that the choice of c# was a very stupid one [11:27] Vincent.Sylvester @hg.zetaworlds.com:8002: At that point might as well rewrite in C directly and throw the framework dependency out the window, but man does that sound like fun... not [11:27] Ubit Umarov: it was very stupid, but too late to change [11:28] Ubit Umarov: ofc to start such thing in c++ with multi.os support was a lot harder to do [11:28] Gavin.Hird @grid.xmir.org:8002: someone needs to download the SL server code :-) [11:29] Ubit Umarov: so possible no one would had done it [11:29] Ubit Umarov: without the c# original promise to run (badly) on windows and linux [11:29] Andrew Hellershanks: I haven't done enough C# programming to realize that you have to use .NET. [11:29] Ubit Umarov: and so far it did.. [11:30] Ubit Umarov: there is no c# without .net [11:30] Ubit Umarov: basicly [11:30] Ubit Umarov: well at least some core parts of it [11:30] Ubit Umarov: we can not use linq for example [11:31] Vincent.Sylvester @hg.zetaworlds.com:8002: Only option is mkbundle, stuff .net into the app, which blows its size up and isn't guaranteed to work either, especially not at OpenSim complexity levels [11:31] Ubit Umarov: but well academic talks [11:31] Ubit Umarov: its c# now [11:31] Ubit Umarov: just a hard crossroad ahead [11:31] Ubit Umarov: but again, no need to rush it [11:31] Vincent.Sylvester @hg.zetaworlds.com:8002: Well hard if MS decides to continue to ruin the fun [11:31] Ubit Umarov: the more we wait the easier is should get [11:31] Ubit Umarov: hopefuly [11:32] Ubit Umarov: bc others will do it :p [11:32] Vincent.Sylvester @hg.zetaworlds.com:8002: I'm crazy enough to have hope they eventually grow a brain [11:32] Andrew Hellershanks: :) [11:32] Ubit Umarov: all software is getting more insane [11:32] Vincent.Sylvester @hg.zetaworlds.com:8002: I blame javascript [11:33] Ubit Umarov: and ppl will be forced to rewrite things [11:33] Andrew Hellershanks: hehe [11:33] Ubit Umarov: well don't know [11:33] Ubit Umarov: just guess we have like 2 years to see the .net6 issue [11:34] Ubit Umarov: possible more [11:34] Ubit Umarov: about opensim [11:34] Ubit Umarov: not many changes last week, i think [11:34] Ubit Umarov: in fac i almost did release 0.9.2.1 today [11:35] Ubit Umarov: the date 22-2-22 was a fun release date [11:35] Vincent.Sylvester @hg.zetaworlds.com:8002: hehe [11:35] Vincent.Sylvester @hg.zetaworlds.com:8002: I didn't even notice that [11:35] Andrew Hellershanks: Just one change this past week to the profiles module and a fix to a log message. [11:36] Ubit Umarov: well i did update libomv with a bit faster OSD to json code [11:36] Andrew Hellershanks: For me it would be a 2-22-22 release date. Not quite as fun but almost. [11:36] Ubit Umarov: and that was basicly it [11:36] Kayaker Magic: I have a question about de-duplication. Does OpenSim do this on images? I hear Kitely does. [11:37] Ubit Umarov: sounds you need to wash those ears to get full indo [11:37] Ubit Umarov: and info [11:37] Ubit Umarov: :p [11:37] Vincent.Sylvester @hg.zetaworlds.com:8002: Dedupe on assets is done hash based, if the hash is the same, the item is just linked via the DB, dynamic textures and such are regionside so they are cleared anyways [11:37] Andrew Hellershanks: Kayaker, There is an asset system that eliminates duplications. [11:37] Ubit Umarov: fsassets [11:38] Ubit Umarov: and xassets [11:38] Ubit Umarov: or how its called [11:38] Vincent.Sylvester @hg.zetaworlds.com:8002: sras and phpsas do it too :) [11:38] Andrew Hellershanks: Yup, fsassets. [11:38] Andrew Hellershanks: SRAS came before fsassets. [11:38] Ubit Umarov: yes fsassets is a c# version of sras [11:38] Ubit Umarov: basible [11:38] Ubit Umarov: and basicly [11:39] Ubit Umarov: xassets does it, still storing all on mysql [11:39] Ubit Umarov: well still marked experimental [11:39] Kayaker Magic: When is the hash on images calculated? After decoding images into raw bitmaps? [11:40] Ubit Umarov: sha256 [11:40] Ubit Umarov: the SL asset data [11:40] Ubit Umarov: and exactly as SL asset [11:40] Ubit Umarov: a fun thing ppl talking about nrew viewers forget [11:41] Ubit Umarov: we only have assets in SL format [11:41] Kayaker Magic: So after compressing them for the asset database? Is that zip or are images just jpg2000 compressed? [11:41] Ubit Umarov: any convertion to other formats is just impossible to do in real time [11:41] Ubit Umarov: unless on small prof of concept things [11:41] Ubit Umarov: actually not need, bc no concept to prove [11:42] Ubit Umarov: to use other formats, the asset storage will need to change to add them [11:43] Ubit Umarov: well the avination unreal viewer was using SL assets formats [11:43] Misterblue Waves: ya... new asset server. New formats, new access permissions, scalable... another whole project [11:44] Ubit Umarov: yeah [11:44] Misterblue Waves: in existing viewers the conversion from SL asset format to a usable format is all done in the client [11:44] Ubit Umarov: a asset UUID will need to point to several formats [11:44] Ubit Umarov: like a texture in jp2000 and dds [11:45] Misterblue Waves: maybe make it an nginx plugin [11:45] Ubit Umarov: well and do that FAST [11:45] Ubit Umarov: the FAST part is something all forget [11:45] Ubit Umarov: and without it, it is useless [11:46] Ubit Umarov: we all know any viewer can be made... [11:46] Ubit Umarov: the unreal one was should most of a region [11:46] Ubit Umarov: at 6fps !!! [11:46] Ubit Umarov: so a nice work... useless.. but nice [11:47] Ubit Umarov: fs was showing same region with 40 fps and doing all things [11:47] Ubit Umarov: see what i mean? [11:48] Ubit Umarov: err would help if i did type right :p [11:48] Ubit Umarov: the unreal one was SHOWING a region at 6fps :) [11:48] Ubit Umarov: still no terrain, water or avatars [11:48] Vincent.Sylvester @hg.zetaworlds.com:8002: Meanwhile new FS release is looking like it will improve performance by a good 10% from what I can tell, I don't recall ever getting more than 60fps here [11:49] Ubit Umarov: ll is improving performance [11:49] Ubit Umarov: they have another batch of changes for a future viewer [11:49] Ubit Umarov: fs is adding those and a few own [11:50] Ubit Umarov: ofc it does depend on region creators [11:50] Ubit Umarov: you will need to pay atention to region build, if you want fps, also [11:50] Ubit Umarov: and have customs police :p [11:51] Ubit Umarov: not letting avatars with complexity off 700000 to get in [11:51] Ubit Umarov: :) [11:51] Andrew Hellershanks: :) [11:52] Ubit Umarov: so what news do you bring ? [11:52] Ubit Umarov: about opensim.. [11:52] Andrew Hellershanks: No news is good news? ;) [11:53] Andrew Hellershanks: Will there be a release today? [11:53] Ubit Umarov: if about ukraine yes [11:53] Ubit Umarov: nah guess i will not do it today [11:54] Ubit Umarov: osgrid also did none on 0.9.2.1 basicly [11:55] Ubit Umarov: but diva is waiting for it to release her code [11:56] Ubit Umarov: i had promissed her a reease in Jan [11:56] Ubit Umarov: and we are almost in march :) [11:56] Gavin.Hird @grid.xmir.org:8002: but what year? [11:57] Ubit Umarov: yeah i didn't told her the year.. but she assumed 2022 :) [11:57] Gavin.Hird @grid.xmir.org:8002: :-) [11:57] Ubit Umarov: so i can't use that trick on her :p [11:58] Ubit Umarov: well and the users that are asking for the code [11:58] Gavin.Hird @grid.xmir.org:8002: they must be begging, asking will not do [11:59] Ubit Umarov: :) [11:59] Andrew Hellershanks: Wow. This hour has just flown by. We are at the top of the hour already. [12:00] Andrew Hellershanks: Any final questions or comments for today before we wrap up this gathering? [12:02] Andrew Hellershanks: Kayaker, fyi, in a large database a hash based asset storage system can reduce the asset table substantially. I've seen anywhere from 25% to 50% reductions in table size. [12:02] Ubit Umarov: sadly if table size means nothing :p [12:03] Ubit Umarov: ofc tables size is smaller, just because the asset data is not stored on it, but on a disk file [12:03] Kayaker Magic: I was curious that hashes of images are so likely to be different for similar images. [12:03] Andrew Hellershanks: yes, but it is a substantial savings in used disk space. [12:04] Kayaker Magic: And I don't trust jpg to produce the same imagage from the same data 2ice in a row... [12:04] Ubit Umarov: guess a lot of ppl uploads the same things [12:04] Ubit Umarov: well it is not jpg, it is jpeg2000, a bit diferent [12:05] Ubit Umarov: and with paramenters fixed on viewers [12:05] Ubit Umarov: so end data may be more identick [12:05] Ubit Umarov: other wise, the hash is just waste of time [12:05] Ubit Umarov: same for mesh.. [12:06] Ubit Umarov: hash only does anything if mesh is upload with same paramenters and identical viewer [12:06] Ubit Umarov: so finla bits are exactly the same [12:06] Ubit Umarov: or then again hashing is uselsss [12:07] Ubit Umarov: so in fact o do have doubts on how effective it really is [12:07] Ubit Umarov: but ppl tell it is.. so.. one must believe,, [12:08] Ubit Umarov: and good viewers do not add Cretion date and uploader uuid to the binary data [12:08] Ubit Umarov: assuming region code will do it ( and does not ) [12:08] Ubit Umarov: if mesh where as spec, the hash would fail always [12:09] Kayaker Magic: Some sort of hash that only looks at the image data, or vertex data of mesh would be nice. [12:09] Misterblue Waves: gonna run in RL.. will be here virtually [12:09] Ubit Umarov: no [12:09] Ubit Umarov: hash only looks to raw asset data [12:09] Andrew Hellershanks: ok, Misterblue. [12:09] Ubit Umarov: cna't look to asset details [12:09] Vincent.Sylvester @hg.zetaworlds.com:8002: Somewhere in this mess someone set xml utf16 for linksets even if all data is utf8 -.- [12:09] Ubit Umarov: no time for that [12:10] Ubit Umarov: nahh think most are actually utf8 telling they are utf16 [12:10] Gavin.Hird @grid.xmir.org:8002: are you sure it cannot look at asset data? [12:10] Ubit Umarov: not was it is done [12:10] Vincent.Sylvester @hg.zetaworlds.com:8002: Yeah that's what I mean, the xml header claims utf16 for no reason, threw me off at first [12:10] Ubit Umarov: and no.. would be very slow to look per asset type [12:10] Gavin.Hird @grid.xmir.org:8002: on postgres I have a tool that when looking at single records will decode the images in the asset table on the fly [12:11] Gavin.Hird @grid.xmir.org:8002: they are just stored in a blob filed [12:11] Ubit Umarov: even doing sha256 is not that fast [12:12] Ubit Umarov: some cpus do have hardware help for that.. not sure .net uses [12:12] Gavin.Hird @grid.xmir.org:8002: you only have to do it on first write in that table [12:12] Ubit Umarov: no [12:12] Ubit Umarov: on any upload [12:12] Gavin.Hird @grid.xmir.org:8002: sure, but uploads does not happen that often [12:13] Ubit Umarov: and does not matter [12:13] Ubit Umarov: any bit changed means other hash [12:13] Andrew Hellershanks: yup [12:13] Ubit Umarov: for now we are not filling the creation date, and creator id of mesh spec [12:13] Gavin.Hird @grid.xmir.org:8002: image data cannot be edited in-world, so only uploads needs to be rehased [12:13] Ubit Umarov: so at least that is not a issue [12:14] Ubit Umarov: even so same mesh even with same parameters will upload diferently [12:14] Ubit Umarov: acording to particular viewer [12:14] Gavin.Hird @grid.xmir.org:8002: yes [12:14] Gavin.Hird @grid.xmir.org:8002: but image data should be fairly constant [12:15] Ubit Umarov: but well as i said seems that hash does savings.. so.. better than nothing [12:15] Ubit Umarov: jpeg2000 constant why? [12:15] Gavin.Hird @grid.xmir.org:8002: there are not many updates to that spec... [12:15] Ubit Umarov: done with diferent versions of openjpeng on kdu [12:15] Gavin.Hird @grid.xmir.org:8002: we don't use kdu [12:16] Gavin.Hird @grid.xmir.org:8002: no viewer does for opensim [12:16] Ubit Umarov: thin lower precision bits [12:16] Ubit Umarov: just think... [12:16] Ubit Umarov: thin fs payed kdu [12:16] Ubit Umarov: think [12:16] Gavin.Hird @grid.xmir.org:8002: nobody are interested in paying thousands of dollars for a KDU lisence for opensim [12:17] Gavin.Hird @grid.xmir.org:8002: fs got KDU from LL [12:17] Ubit Umarov: it is not for opensim [12:17] Ubit Umarov: is for all [12:17] Gavin.Hird @grid.xmir.org:8002: Kokua paid for [12:17] Andrew Hellershanks: kdu? [12:17] Ubit Umarov: viewers do not pay kdu for sl or opensim [12:17] Ubit Umarov: that is not like havok [12:17] Gavin.Hird @grid.xmir.org:8002: they pay for KDU [12:17] Ubit Umarov: they pay per viewer [12:18] Gavin.Hird @grid.xmir.org:8002: with no chance of getting the cost covered for opensim, nobody will want to pay for it [12:18] Ubit Umarov: so if they pay "for sl" they payed "for opensim" [12:18] Andrew Hellershanks: I looked up kdu and the top result was General Sir John Kotelawala Defence University. :) [12:18] Ubit Umarov: unlike havok, where they just use LL license [12:18] Gavin.Hird @grid.xmir.org:8002: yes, but the only one who have paid for KDU is kokua [12:18] Gavin.Hird @grid.xmir.org:8002 whispers: FS got it from LL [12:19] Ubit Umarov: ok [12:19] Gavin.Hird @grid.xmir.org:8002: apart from that nobody use it because of cost [12:19] Ubit Umarov: yeha fs ships with openjpg for opensim [12:19] Gavin.Hird @grid.xmir.org:8002: yes [12:20] Gavin.Hird @grid.xmir.org:8002: not allowed to use the license outside of SL [12:21] Gavin.Hird @grid.xmir.org:8002: so I have had to strip the KDU support out of the 3.0 builds [12:22] Ubit Umarov: well all those diferences make the deduplication fail [12:22] Ubit Umarov: not going to the detail of all cpu FPUs are diferent etc [12:23] Ubit Umarov: intel things have bugs remember ? :P [12:24] Ubit Umarov: so i have no real numbers on how good is fs deduplication good [12:24] Ubit Umarov: hope it does dedupe the "new script" :p [12:25] Vincent.Sylvester @hg.zetaworlds.com:8002: Good question, I know it certainly doesn't do notecards, cause they have creation date and such saved in their data so the hash is always different [12:25] Ubit Umarov: yeah [12:25] Vincent.Sylvester @hg.zetaworlds.com:8002: I wrote an article on the garbage OpenSim collects on these things [12:25] Ubit Umarov: some assets have date etc [12:25] Ubit Umarov: mesh where suposed to have [12:26] Ubit Umarov: but it is server job to add those, and we never did [12:26] Ubit Umarov: not sure what viewers add to the jpeg2000 metadada [12:27] Ubit Umarov: possible also date and creator there, making ghe hash useless on mesh [12:27] Ubit Umarov: on texture i mean [12:27] Ubit Umarov: wel easy for you to check if using fsassets :) [12:28] Ubit Umarov: upload same textyre twcie and compare the hashs [12:29] Ubit Umarov: not sure if who made sras and fsassets knew the metadata viewers add [12:30] Gavin.Hird @grid.xmir.org:8002: I don't think the viewer adds any metadata to the raw imagedata [12:30] Ubit Umarov: well it is j2000 data not raw image [12:31] Gavin.Hird @grid.xmir.org:8002: you understand what I mean [12:31] Ubit Umarov: ok :) [12:31] Ubit Umarov: like crreation data on the info areas of the j2k [12:33] Ubit Umarov: animation seems to be just anim [12:34] Selby.Evans @grid.kitely.com:8002: bye all [12:34] Ubit Umarov: cya [12:34] Andrew Hellershanks: Another half hour has passed. I need to get going so I will close the meeting for today. Thank you all for coming. See you again next week.