The dustmen take to space!

[FeedBot]

The dustmen take to space!


Hey everyone! This is Prism X, EVE's database developer, with his first ever Dev Blog. I do not get to do many of these as most of my job revolves around the things behind the scenes, but I reckon there might be a second one someday. As CCP grows in size more and more people with database experience are drafted into our ranks and my focus thus shifts elsewhere where it is required. Currently it is shifting towards announcing one of the features in the upcoming Trinity 1.1 update. The cleanup of space-junk!

<p>The How

<p>The functionality itself depends on the type of space junk in question.

<P><B>Generic Junk</B>

<p>The following item groups are defined as generic junk: Drones, Fighters, Shuttles and Rookie Ships. Generic space junk will be removed (Read: permanently deleted) from open space the first of every month. As generic junk of this kind has been purposely left floating in space for logistical reasons there is an exception to this rule. See below.

<P><B>Anchorable Containers</B>

<p>Anchorable containers will from now on have a 30 day lifetime from last usage. This means that once you anchor a container it will gain an "Expires In:" attribute. If this container enters downtime with 0 days left it will be removed from space. Last usage is either when it was last anchored or opened. This means that as long as you take care of your space cache every month, it will be safe. However, if you leave an unanchored container it will be removed immediately the next down time. The same exception applies to anchorable containers as to generic junk (see below).

<p><B>Examples:</B><BR>
And here is an unanchored giant container one.<BR>
<a href="http://www.eve-online.com/bitmaps/devblog/containerExpiry1.png"<img border=0 src="http://www.eve-online.com/bitmaps/devblog/containerExpiry1.png"></a>;
<P>Here we can see a recently anchored small container<BR>
<a href="http://www.eve-online.com/bitmaps/devblog/containerExpiry2.png"<img border=0 src="http://www.eve-online.com/bitmaps/devblog/containerExpiry2.png"></a>;

<P><B>The Starbase Factor</B>

<p>To avoid disrupting logistics as much as possible, we decided to give Starbases a protective field from all Fedos, Dustmen and other factors which might remove your stuff. This means that both generic junk as well as the anchorable containers will not be removed if left in close proximity to Starbases. This is regardless of who owns the item, but keep in mind that leaving things outside enemy Starbases is probably not a good way to safeguard them and their content. This setup should allow for most heavily used item caches to remain useful, hidden caches will have to be maintained and abandoned caches will leave the game freeing up resources for us all.

<p>The Why

<p>Ever since we announced the Need For Speed (NFS) initiative we've been searching every nook and cranny for refactoring possibilities. However, it's not just about the refactoring of old features but also a frame of mind when coding the new features or, as in this case, it can be the driving factors behind the feature itself. We are all quite aware of all that space junk which never seems to go away. Granted, it's never a cache of Fighters we stumble across but who doesn't remember that anchored container in their old high-sec home? Yeah, the one that read "Prism X cna has Chezeeburgers!! ... Plz send ISK!" or something similarly witty. Remember how annoying you found it?

<P>Not surprisingly, it turns out to be more than just an annoyance. I don't want to go into too many details (though I know some of you want me to) but it is common knowledge that upon entering any system you preload all (non-sensitive) information needed about the system from the node it's on. As this is frequently queried information the node caches - in pre-cache or on-demand cache depending on information - all this information so it does not need to query the live DB repeatedly for the same information.

<P>So you jump into a system, during the session change all the information you need is transmitted to you from the SOL nodes different caches. This may include refreshing some parts of the caches from the live database as they may have been invalidated or flushed to make room for new, possibly more frequently used, items. Currently, all is well in space as you are the only living entity in this over-simplified example and every solar system you visit has a dedicated node. (Side note: In this example caching is more for 're-load' than 'pre-load' purposes).

<P>Now, if we move back to reality and look at the actual scale of things and accept uncomfortable things such as limitations, things pan out a bit differently. You are now a part of approximately 30K players constantly moving between systems, and nodes, thus demanding new set of system information with every jump. Due to resource limitations we have many systems on one SOL node and only limited cache sizes for all the different caching services. Caches now get invalidated more frequently (Note: <I>not all do, but due to limitations the less mission critical ones must</I>) with more players requesting information from larger datasets.

<P>Now let's imagine what happens to this information transfer if one system alone on the SOL node were to contain more cargo container information than can exist in the container cache at any given time (<I>Note: this is simplification again. The nature of caching mechanisms is that you can never cache everything so this does apply in a more complex sense.</I>). This means that whenever someone jumps into that system each and every cached cargo container on every other system on the node is invalidated to make room for the new info. Furthermore the system cannot cache in all the information in one go and, depending on implementation of the caching service, this can mean a whole bundle of redundant I/O's.

<P>Now imagine all those containers are somebody's forgotten containers and they are causing your session changes to last longer, our database to be queried more frequently, more data to go over the wire etc. For no good reason, nobody is using them! This is the NFS initiative in a nutshell. With this implemented we're effectively increasing the container cache efficiency rather than just its capacity. Naturally increasing the capacity is not out of the question but the gain from a capacity increase is proportional to its efficiency.

<p>Hence: <I>The Space Junk must die!</I>

<p>The When

<p>Like I mentioned earlier this is to be implemented in Trinity 1.1.

<p>In the future we might tweak the expiry time on anchored containers, and depending on performance gain we might schedule the junk cleanup more frequently than once per month. Perhaps some sort of monthly anchoring charge would be in order. Nothing further is set in stone though. Nothing other than: <I>The Space Junk must die!</I>



http://myeve.eve-online.com/devblog.asp?a=blog&bid=540