| Author |
Seperate servers for Planet and Ship based calculations? |
Fluttershy
Fleet Admiral
Joined: September 24, 2011
Posts: 778
From: Fluttershy
|  Posted: 2013-01-29 15:13
DS I imagine is extremely CPU heavy for the servers.
Would it help performance, stability, and lag if ship to ship interactions were handled on a different server?
One server would process all planetary happenings and update the client when necessary.
The second server would process all player locations, weapons fire, damage dealing, collisions, and ship AI.
I don't know how you have things set up currently, or if this would help.
If one aspect of the game tends to saturate the processor usage on one server, I wonder if it is best handled independently from the other aspects.
Again, this might not even be a problem, but the game does seem to become very unstable when you got maybe a dozen players online and fighting.
_________________
|
Pantheon
Marshal
Palestar
Joined: May 29, 2001
Posts: 1789
| Posted: 2013-01-29 16:27
Multi-threaded calculations are an issue due to syncronisation. This is why many games, even now, are still single threaded. What you usually see is a lot of helper threads doing work on other processors - but the main thread, which contains all the timing and heavy work, remains as a single thread. This is the same for pretty much every MMO. EVE Online, World of Warcraft - you name it, it's single threaded (even if it looks like it's pulling load across all processors, in the config files you'll often see a line along the lines of "MainThreadProc=#", which indicates which processor the main thread is running on - no helper threads may run on this processor).
Offloading to another 'server' (which is essentially just multi-threading - we only have one server, even though it has 8 cores - each has an instance of the game locked to its own core) breads issues with syncronisation which would could effectively make all the work to offload it null and moot.
The best action to take would be to offload all the AI functions into a seperate thread and treat it as a local player of sorts, and to lower the number of planets per map so we can dynamically sleep maps with nothing going on (lowering its active simulation rate, so that it keeps in sync with the rest, but only simulates the world at half the rate so that another map on the same core which may be under load can have more room to breath).
_________________
|
White-Knight
Admiral
Pitch Black
Joined: January 12, 2013
Posts: 12
| Posted: 2013-01-29 16:46
Although synchronisation is an issue, mutual exclusion and condition synchronisation in threads can be used to allow some features to be run on seperate cores (Multi-threading).
It is not quite true that EVE Online is single core; Seperate cores run seperate services (microthreads). Using techniques such as Stackless python and mutual exclusivity to avoid errors such as interleaving in threads.
As quoted from this dev blog:
(For the purposes of explaining the concept, not advertising: http://community.eveonline.com/devblog.asp?a=blog&bid=286)
'Why isn't EVE a multithreaded "application"?
First off, EVE is a very very multithreaded application and far more so than most you will encounter. We use Stackless Python to get microthreading abilities within the single process running on each CPU.
To better clarify that, right now, the EVE cluster is running 110 threads on 110 CPU's. Within those threads we have 100's to 1000's of microthreads running various services.
Other MMO's are usually doing a monolithic design (one big supercomputer) which do not scale very well or they have a distributed design but have physical assignment of threads to CPU's where a thread is running a geographical part of the universe but do instancing which they can run abstract from the geography because the instanced location has nothing to do with the geography and thus does not matter to the universe. '
[ This Message was edited by: White-Knight on 2013-01-29 16:50 ]
_________________
|
Fluttershy
Fleet Admiral
Joined: September 24, 2011
Posts: 778
From: Fluttershy
| Posted: 2013-01-29 16:47
I suppose the question I should be asking is, what causes the greatest load on the server?
It could be anything as far as I know, since it all depends on how well optimized it is.
Like, I don't know how often the AI checks its surroundings, what it checks for, and how far.
_________________
|
Pantheon
Marshal
Palestar
Joined: May 29, 2001
Posts: 1789
| Posted: 2013-01-29 17:22
Quote:
|
On 2013-01-29 16:46, White-Knight wrote:
|
|
Python is considerably behind C++ in terms of performance, and partly the reason why CCP are re-writing a lot of performance heavy operationsin C++ to gain performance. We have this huge internal requirement for performance. Everything we do, everything we touch, must have as little impact on the server as possible, which is why we are able to do so much in DarkSpace.
All of DarkSpace is written in C++, and extremely optimised, and whilst I'm sure we'd gain a lot from implimenting what you're suggesting - our time is very precious and little, so spending it on splitting the map up which is far easier will give us a lot more breathing room and allow us to stave off having to re-write a significant portion of how our backend works.
Quote:
|
On 2013-01-29 16:47, Fluttershy wrote:
I suppose the question I should be asking is, what causes the greatest load on the server?
It could be anything as far as I know, since it all depends on how well optimized it is.
Like, I don't know how often the AI checks its surroundings, what it checks for, and how far.
|
|
Greatest load depends on a lot of things, like how many of something are active or alive at that period of time, so it's all very relative, however I can give you the highest costs:
1) AI
2) Smart nouns
3) Noun simulation
AI is a no-brainer. I mean, it's AI. Smart nouns are missiles and fighters - they're somewhere inbetween normal stuff and AI. Not quite as advanced, but they require some real time calculations like trajectory adjustments and stuff.
"Noun simulation" is very generic, but I'll go into a bit more detail about it. Every object in DarkSpace is what we referr to as a noun, and all nouns have at least some degree some level of logic to simulate. A structure is a noun, a planet is a noun, a ship is a noun - some things require more, some require less processing time.
Planets used to be a massive burden because we had 32 structures per planet (doesn't take a genius to do 32 x number of planets = a lot), but also because a lot of those structures were defence bases which required significant extra logic to do their job (scan for incoming stuff to shoot down/at, etc). We had to make changes to change this because it was getting to the point where the server sitting at idle was putting about at around 70% load.
After the defence changes we got this down to 30% - which is a significant saving (and why we can't go back to how it was, no matter how much people want it).
When fights start appearing and projectiles start flying, it's not too uncommen to see the server up around 60% nowadays but it's VERY rare to see the server go over that unless I or another admin spawns 20 or so AI, at which point the server will usually shoot up above 75% which automatically triggers the AI to go into stupid mode, which means they update less frequently (which is why I don't spawn too many anymore).
Bit of extra info : there used to be an AI written in LUA by Chromix which was utterly savage and was basically unkillable (required 6 players to kill it). It scanned its surroundings and incoming projectiles with 100% precision so could dodge everything, and could calculate which path would incure the least damage. Only problem is you could only spawn one, as when you did, it pegged the server at 100% CPU usage (part of that is because it was written in LUA though).
We have a lot in place to ensure the server rarely ever peaks out, and to my knowledge it hasn't even come close to that under purely player load.
_________________
|
Challenger
Marshal
Joined: March 28, 2002
Posts: 886
| Posted: 2013-01-29 19:50
I know that to a lot of people AI are important because of pres farming but keeping an open mind here, can we get rid of a lot of the AI? Have the planets update resources as if trannies were shuttleing goods but just not have the ships in the actual game. I would say the only AI to keep would be Engineers and supply ships. Even with those we can dumb them down, strip their weapons, limit them to basic functions only.
I know we are talking about a significant change but maybe it is worth considering? If this were to happen how would it help the server load you think?
Bringing this topic into another, would be advertising to get more players. If we can increase the server load by eliminating AI so that we can get more actual players in here with a significant decrease in lag then I would say that is worth the AI sacrafice.
Also a benefit people complain about how everyone is a high rank and it is tough for noobs. Granted some people put time and actual effort into the game and deserve their rank but there are a lot of others who pres farm. Don't bother saying NO because profiles say it all.
I don't know, just throwing an idea out there.
_________________
|
Walrus of Apathy
Admiral
Templar Knights
Joined: August 07, 2005
Posts: 466
From: Dorans Basement
| Posted: 2013-01-29 20:18
Get rid of all the AI and people complain there's never anything to shoot at.
Keep the AI and people complain they're a nuisance and get in the way.
Oy vey.
Personally, I never liked the AI and I do feel that there are too many in the MV. But the AI do serve a purpose and eliminating all of them would take away from the game.
Quote:
|
I would say the only AI to keep would be Engineers and supply ships.
|
|
Heh, those are the ones I'd like to see removed out of all of them. Both oof them automate services that should be done by players. Construction and Supply are great way for new players to get lots of Pres relatively quickly for a minimum of effort until they can get to the ranks where the combat ships start being effective. A lot of players, CoughBardicheCough, like Construction and I hate seeing all that done by AI.
And the AI Sups are just the worst. I had one follow me in a battle the other day and I spent more time trying to get the dumb thing out of my line of fire than I did firing at the enemy.
Quote:
|
Bit of extra info : there used to be an AI written in LUA by Chromix which was utterly savage and was basically unkillable (required 6 players to kill it). It scanned its surroundings and incoming projectiles with 100% precision so could dodge everything, and could calculate which path would incure the least damage. Only problem is you could only spawn one, as when you did, it pegged the server at 100% CPU usage (part of that is because it was written in LUA though).
|
|
Heheheheh, if you could find that and fix it up so it doesn't kill the server so much, that would make a pretty sweet event out of it.
_________________
|
Pantheon
Marshal
Palestar
Joined: May 29, 2001
Posts: 1789
| Posted: 2013-01-29 20:28
Quote:
|
On 2013-01-29 19:50, Challenger wrote:
I know that to a lot of people AI are important because of pres farming but keeping an open mind here, can we get rid of a lot of the AI? Have the planets update resources as if trannies were shuttleing goods but just not have the ships in the actual game. I would say the only AI to keep would be Engineers and supply ships. Even with those we can dumb them down, strip their weapons, limit them to basic functions only.
I know we are talking about a significant change but maybe it is worth considering? If this were to happen how would it help the server load you think?
Bringing this topic into another, would be advertising to get more players. If we can increase the server load by eliminating AI so that we can get more actual players in here with a significant decrease in lag then I would say that is worth the AI sacrafice.
Also a benefit people complain about how everyone is a high rank and it is tough for noobs. Granted some people put time and actual effort into the game and deserve their rank but there are a lot of others who pres farm. Don't bother saying NO because profiles say it all.
I don't know, just throwing an idea out there.
|
|
The server will never lag due to AI. It has the ability to scale the AI logic simulation if the server load goes above a certain percentage. Like I said above, there has not been a situation which has maxed out our new server.
This is taken from the Sagittarius log:
01/29/13 21:22:32 | STAT | WorldServer | Received: 16.7 k : 0.3 k/s, Sent: 1670.3 k : 27.8 k/s, Load: 0.48
This is taken from the Lacerta log (to give a comparison about how the map can affect things greatly, and Lacerta isn't small remember, but it is when compared to Sag):
01/29/13 21:29:30 | STAT | WorldServer | Received: 1.2 k : 0.0 k/s, Sent: 444.0 k : 7.4 k/s, Load: 0.26
You can see Sag is barely even hitting 50% whereas Lacerta is 26%, and that's with a map that's ridiculously large, has a lot of AI, a lot of planets, and players. The only 'problem' we have, is that when we want to utilise AI, it tends to hit the 75% very quickly. However, you have to understand that Sagittarius sits around 35% usage at idle, with no-one in it and nothing going on, so cutting up the map will go a great way to helping things along.
If you got 80 players into Sagittarius, and the CPU load without AI went over 75%, the AI would stop completely and just sit there doing nothing. They're not taking up resources as we don't update any of the logic if the load goes that high.
This is far different than a case where the load is high because of the AI, at which point you have a fluxuating load due to the fact that some AI will turn off, the load will go below 75%, they will turn on again, it will go above 75%, and other AI will turn off. This is why they can sometimes appear as if they're a bit 'stop and go' with what they want to do. But again, if the load was above 75% due to players, the AI would just stop alltogether, and wouldn't impact things much at all.
[ This Message was edited by: Pantheon on 2013-01-29 20:31 ]
_________________
|
Talien
Marshal
Templar Knights
Joined: May 11, 2010
Posts: 2044
From: Michigan
| Posted: 2013-01-29 20:38
Quote:
|
On 2013-01-29 20:18, Walrus of Apathy wrote:
Quote:
|
I would say the only AI to keep would be Engineers and supply ships.
|
|
Heh, those are the ones I'd like to see removed out of all of them. Both oof them automate services that should be done by players. Construction and Supply are great way for new players to get lots of Pres relatively quickly for a minimum of effort until they can get to the ranks where the combat ships start being effective. A lot of players, CoughBardicheCough, like Construction and I hate seeing all that done by AI.
|
|
Yeah. As someone who enjoys building it really bugs me when people bomb and cap a planet then don't bother building on it or even locking it then AI come in and make a mess of things. I may enjoy building a planet that's not a steaming pile of fertilizer, but half the time I go to fix a planet with existing structures I actually lose pres because I've had to scrap so much stuff first. Of course, this happens with planets that other people have built too, but part of the problem is some people see AI builds and think they must be good so they imitate it.
Ditto on the supply ships, they often get in the way.
What I'd like to see would be no AI present unless there are players actively engaged in combat and not just sitting idle waiting for something to happen. If you attack a planet then AI spawn and move to defend it instead of trying to get there from wherever they happen to be like they do now. If a player sends a distress, AI spawn and move to the combat area to provide support but retreat to the nearest gate and despawn once there are no more targets instead of staying in the server.
_________________
Adapt or die.
|
Hyperforce
Grand Admiral
K'Luth Revolution
Joined: January 25, 2009
Posts: 43
From: Perth, WA
| Posted: 2013-01-29 21:23
it appears my first intial thoughts on a server upgrade isnt so straightforward it seems like the only way to correct this cpu intensive calculations i think would be as white-knight suggested but it this would mean almost a complete rewrite of a huge amount of code for the game so i think u guys have a big tast ahead of you and when ds gets a lil big and it will with the right decisions u will be forced to do this drastic overhaul someday sooner than u would like...i suppose im optimistic towards DS' future
_________________
|
Pantheon
Marshal
Palestar
Joined: May 29, 2001
Posts: 1789
| Posted: 2013-01-29 21:27
Quote:
|
On 2013-01-29 21:23, Hyperforce[MBS] wrote:
it appears my first intial thoughts on a server upgrade isnt so straightforward it seems like the only way to correct this cpu intensive calculations i think would be as white-knight suggested but it this would mean almost a complete rewrite of a huge amount of code for the game so i think u guys have a big tast ahead of you and when ds gets a lil big and it will with the right decisions u will be forced to do this drastic overhaul someday sooner than u would like...i suppose im optimistic towards DS' future
|
|
It's all about getting the most bang for buck, and in this regard, 'buck' is how much time we have to spend on working on the game (we all do it in our free time).
The most we can get for the least work, is to re-design the map so that we don't have ridiculously large maps. The difference between Sag and Lac is staggering (25% load difference). 25% is not a small amount. Re-writing how the entire backend server works would take considerable time, and that would take away from everything else we need to do.
_________________
|
Koda
Marshal
Fatal Squadron
Joined: August 29, 2002
Posts: 1384
| Posted: 2013-01-29 22:19
Some really great ideas floating around these threads.
I would gladly give up 50k credits to see that perfect AI that was previously mentioned.
Quote:
|
There is nothing more noble and beautiful than a warrior with no distractions. One could say he is the closest thing to God.
-Treize Khushrenada
|
|
I would gladly fight this EPYON of yours..
-Charz
[ This Message was edited by: Charz on 2013-01-29 22:29 ]
_________________
|
Two Face
Marshal
Joined: November 30, 2009
Posts: 145
| Posted: 2013-01-29 22:24
Quote:
|
On 2013-01-29 19:50, Challenger wrote:
I know that to a lot of people AI are important because of pres farming but keeping an open mind here, can we get rid of a lot of the AI? Have the planets update resources as if trannies were shuttleing goods but just not have the ships in the actual game. I would say the only AI to keep would be Engineers and supply ships. Even with those we can dumb them down, strip their weapons, limit them to basic functions only.
I know we are talking about a significant change but maybe it is worth considering? If this were to happen how would it help the server load you think?
Bringing this topic into another, would be advertising to get more players. If we can increase the server load by eliminating AI so that we can get more actual players in here with a significant decrease in lag then I would say that is worth the AI sacrafice.
Also a benefit people complain about how everyone is a high rank and it is tough for noobs. Granted some people put time and actual effort into the game and deserve their rank but there are a lot of others who pres farm. Don't bother saying NO because profiles say it all.
I don't know, just throwing an idea out there.
|
|
+1
_________________
|
DiepLuc
Chief Marshal
Joined: March 23, 2010
Posts: 1187
| Posted: 2013-01-30 13:48
Quote:
| On 2013-01-29 20:28, Pantheon wrote:
This is far different than a case where the load is high because of the AI, at which point you have a fluxuating load due to the fact that some AI will turn off, the load will go below 75%, they will turn on again, it will go above 75%, and other AI will turn off. This is why they can sometimes appear as if they're a bit 'stop and go' with what they want to do. But again, if the load was above 75% due to players, the AI would just stop alltogether, and wouldn't impact things much at all. |
|
This is why players demand cutting off AI's number.
Not many players read this thread, hence they don't understand why AI becomes so useless suddenly for some times.
Players don't care of server's CPU usage, they care of game experience. Clearly when you join a game and you see stupid AI doing nothing, although they're rich in number, you'll get mad. I can list some situations:
- There are many combat AI but the supply doesn't show up when you need help.
- The supply platform doesn't repair you when you're close to it.
- You're bombing planets but the school of combat AI are idling far away.
That's why we suggest scale the map smaller and cut off AI number. By that, less AI but firmer intelligence.
I insist that player faction AI should be cut down but increase non-playable AI like pirate & MI (remove death beam, for godsake).
I also recommend to eliminate the AI transport resource between starports. They were there since players whined at ship cost. But I think a better solution is to increase mine's production. There should be a kind of mine planet and players should get resource from it to feed the shipyard. With AI resource transport deleted, combat AI number can stay un-touch I guess. After all, you need to spawn ship, you have to carry resource yourself.
_________________
|
Ravendark
Marshal
Sanity Assassins
Joined: July 01, 2010
Posts: 443
| Posted: 2013-01-30 14:53
+1 to removal of builder AI. man are they dump as #&$/(%!
+1 for rewriting the 'super AI' to more server friendly code and make some badass events for reaching servers costs (we have to burn out that server if we're paying for it! xD joke...)
_________________
|
|
English
Deutsch
Nederlands
)
