Brad Wardell's views about technology, politics, religion, world affairs, and all sorts of politically incorrect topics.
Practical realities of adaptive software development
Published on March 31, 2006 By Draginol In Game Journals

Stardock's business model has been predicated adapting to what users request.  The company's motto is "Innovation on demand".

The idea being that we release the best product we can and then we rapidly enhance it based on user feedback.  Users of Object Desktop, for instance, are very familiar with this.  We have also introduced this model to our game development which has had very positive results.  Our conclusion isn't so much that Stardock is "great" at customer service but rather most PC game companies are simply not geared up to do the kind of development model we're used to.

The PC utility market is extremely competitive and so rapid updates are one of our critical advantagers.  Since 1999, a huge percentage of Stardock's R&D budget has been put into building an infrastructure to crank out user requested updates quickly.

Central to that strategy has been Stardock Central which allows, literally, a developer to put up a build onto the system and have it available to users without creating an installer.  In addition, a custom set of forums that interact with the user's account information was created over the past few years that lets them interact with all our sites, Stardock Central, and access things quickly and easily depending on which site they're on.  It also has allowed us to implement various "community" features that helps make users part of the team -- because they are.

Where the model is starting to break down is as we've become more mainstream, the sheer numbers of users has started to show weaknesses in our system.  In addition, the demographic has dramatically changed which has significantly hampered our ability to do what we're best known for doing -- crank out rapid updates.

In the past, especially with Object Desktop, our demographic was almost completely all power users.  As a result, we could release alpha-level updates to those users who, in turn, would let us know what changes they wanted or any bugs they found. Power users who weren't comfortable with alphas would wait for betas and power users not comfortable with either would wait for releases.  Bugs, features, etc. would be queued up to be addressed and the system flowed nicely.

Over the past year or so, the utility market we're in, the desktop enhancement market has had two odd things happen: First, the market has increased in size dramatically. Secondly, the number of other providers of such software has strangely declined (despite more opportunity).  What happened to all these developers is subject for a different topic. But the net result is that WinCustomize.com, Stardock's site for supporting Windows customization, exploded with far more users than had been expected. Even now, it's basically limited by server levels.

The market size increasing though has brought in users who are not power users. Users who don't read manuals, don't understand the difference between alpha, beta, RC, release. Don't know how to resolve problems on their own.  The net result has been that the same updates that we used to just throw out onto Object Desktop (or ThinkDesk) now have to go through an internal QA or else we'll suffer in massive support calls (phone, email, more phone, more email) that drowns out truly serious support (In the last week of February, a sampling of tech support email resulted in the conclusion that roughly 2 out of 3 emails would have been resolved had the user simply looked at the readme in the given product or gone to the knowledge base -- two years ago, that would have been 1 out of 10 emails).

Of course, we're not set up with that kind of internal QA, not in terms of a long-term infrastructure.  When we do a major release like WindowBlinds 5 or Galactic Civilizations II we can put together a pretty impressive QA team using people from other parts of the company. But those people are far too expensive to have doing QA for the long term.  So now we're in the position of having to build a full time QA staff that can rapidly handle the updates that come from the various development teams and get them out there.

Similarly, we're going to have to start hiring people to do forum support. More people I should say.  In our rapid development model, we would put up an update that would list what's in there and users would respond with what they found worked/didn't work.  Now, we put up updates and people will ask "Did you fix <bug X>?" or "When is <Feature Y> going to be put in?"  That is, assuming the user even reads through the change log at all before asking questions.  Then there's the users who read the change log but don't try the update and complain about theoretical things.  "Oh, I don't like that new WindowFX feature, I think it might slow down my computer." or "I think the economic change in GalCiv II is bad" (without having tried it).

Of course, right now, we simply don't have those people to beef up a permanent large scale QA department or to act as a day to day buffer between development and support. They have to be hired and trained which we are, but it's a slow process.

So the business model of developing niche software (desktop enhancement utilities and turn based strategy games) in order to attract power users and hard core games has started to break down.  They have started to break down because those markets are no longer niche markets. Turn based strategy games have become mainstream and desktop enhancements are mainstream. And so we will have to evolve.  But it's been painful and likely to continue to be painful for awhile.


Comments (Page 2)
2 Pages1 2 
on Mar 31, 2006
Send me a collector's edition and a t-shirt and I'll be happy to answer questions on the forums.

Simple answer is to tier the access to updates. I read on the boards someone who was confused because a beta was available and didn' t know what a beta was or if they should be playing it. That kind of person shouldn't even have access to a beta!

You put a beta out there so it can get tested and you can get feedback. Make the beta group voluntary and talk about it only on the forums. If they sign up but ask dumb questions, remove them from the list.
on Mar 31, 2006
#16; You would want a 'stupid' person to have access to a beta because only a 'stupid' person can make 'stupid' mistakes that the application does not handle properly which makes it crash.

Assuming the 'stupid' person is able to document the steps taken, or even better, if the application itself tracked the users actions (Debugging, performance decreasing) and auto-launched a bug report when said crash/bug occured, you'd have a pretty good tester.

Course, then they would complain that their product failed...

So, yes, you want 'stupid' users to find the 'stupid' mistakes you've made in your application, but you dont want them to complain about them afterwards.



on Mar 31, 2006
It's the catch-22 of independent software houses. Due to your small size and nature, you are able to be more agile when it comes to reacting to customer needs. The result is a better product that blows the competition away. So you have a better product, you have better service and then the customers start to flood in (well, more of a gradually growing flow). Eventually you find yourself with a body of users too large to support in the way you have before, so you have to grow. But growth means you start to lose the agility of a small company if you don't do it very carefully. So it raises the question of "Is it actually a good thing to grow beyond a certain point?"

What's the point of diminishing returns? At what point does growing your customer base actually hurt the business you've worked so hard to build? Sure, growing may allow you to release more products, see greater revenue, but is service suffering? Do you lose the close contact you once had with your customers? Is it even feasable after a certain point to try and maintain that personal contact? Does it make sense for a company like Microsoft to have its executives and upper management getting involved on the community front-lines?

Popularity is a double-edged sword as you've said. It requires the company to grow, but growing too fast and too much could kill the culture and community that's been developed over the years.

I'm confident Stardock can handle this growth well, but I don't doubt the transition to a larger company will have it's challenges and growing pains.
on Mar 31, 2006
Brad -

I sympathize with the problems of your success, and as great as that success has been I hesitate to offer any advice, but you should spend a little time reviewing the app descriptions in SDC and looking back through those changelogs you say we don't pay enough attention to. The main description of ObjectBar in SDC still says it supports Win98 & WinME, for instance. I routinely check changelogs before running updates, but over the past couple of years, I'd estimate that there was no changelog information at all about a third of the time (the History link in SDC).

It's not surprising that people don't read those things when they've grown accustomed to them being so often out of date or absent. When the app documentation lags so significantly, which has been a "tradition" at Stardock, the casual users (as opposed to power users) easily get confused & frustrated by the "mismatches," despite the wealth of tutorial information available on WinCustomize. Even the software versioning system SD has adopted has been a point of contention for users, forever it seems. Why the tootip info and Help/About info can't match up with the actual version number is a mystery to the casual user. I don't consider myself a "power user" and my own preference would be to get the app now & documentation later, but that's just me - I suspect that devoting resources to being sure the descriptions & documentation are up-to-date on the front end will significantly cut down on the support demand on the back end.

Thanks for being so open about things and for soliciting user input in so many ways, and do accept this in the spirit intended. I want to see SD improve and succeed, too.
on Mar 31, 2006
lol, you don't have to give me anything and I'll still answer questions on the forums for you. I do that anyways...

But no matter how many I answer, it will never be enough. The problem that I see is how the forums are structured. The blog style interface works fairly well with the smaller community of hardcore followers, who are patient and knows how to work it, but when the user count goes over the roof, server load aside, the sheer amount of topics alone makes it nearly impossible to adress. So while you may answer one question, another one which adresses the exact same problem pops up a while later, because the first one was sufficiently answered and is now buried on page 8... Then, a few hours pass and another topic is created on the same problem, etc...


IMO, the first thing that really needs to be done is organization. I suggest splitting up the sections even more. I have never had problems when going into one of the smaller organized subforums, but on the main GalCiv2 forum, it is a complete mess. Hell, the sheer number of stickies alone could scare a normal person because it would take them days to read through. So here's what I think might help us, help you guys while you are looking for QA staff:

1. Break up the forums a bit further and don't list everything that is posted in the GalCiv2 section show up in the GalCiv2 category. Ideas goes into Ideas, and Modding goes into Modding only. This cuts down the number of topics that shows up and let people choose the types of topic they see, and help keep topics around. Basically adopt a more traditional "forum" style of posts goes into folders, etc... just for organization sake. I wonder how many people actually realize that you even HAVE subforums for ideas and suggestions...

2. Add a question and answer forum/section, with a stickied topic that is edited and maintained pointing to all the topics dealing with frequently asked problems. This should cut the number of posts down by half, if not more. Of course a QA person organizing it would help, but if not I'm sure someone can step up and handle it if you just sticky it for them. A stickied topic explaining how to properly use the forums will help immensely as well. I don't know how many times I've seen people complaining the quote feature is broken.

3. The ability to delete posts/comments. I don't know how many times I've seen people accidentally posted 3 times in a row because the system was slow to respond. This would really help make topics a bit cleaner. It would also reduce useless double topics. Add some moderators to clean the mess that people makes, and it will probably work.


About volunteers administration: I would say no, to a certain extent. This is mainly because I've seen some political bickering among forum administrators and that has never gone well. However, I'm not against a system that would promote knowlegeable and helpful people to a certain status, like "helper" so that when they answer a topic you can be safe to know that they are probably right. These people could be voted for by the community or chosen by the real moderators to help them out.


About betas: I would say don't post this to everyone in the GalCiv2 channels or journals or SDC. That would just lead to every average joe with no understanding of it trying it out. Instead, put up a post in an appropriately named "beta/in developement" section. You could sticky a post of the rules for using the beta as a disclaimer of sorts in the section. That way, only those that really wants it gets it. Tier access to beta updates seems a little extreme, and contrary to Stardock's fair treatment of their customers. But putting it into a "beta corner" and you will probably only have hardcore people playing with it.


Don't get me wrong, I like the fact that the forums can be accessed from all aspects of Stardock, it is an interesting little feature, but the shift to mainstream and the coresponding user base pretty much forces these kinds of changes. You pretty much need an in-your-face list of problems and frequenently asked questions/solutions to help out the new people and keep down the number of repeating topics and posts. Similarly, you also need to hide your betas away from those same people until they are ready. Organization is the key, I think you guys have a really great thing going, and I'm sure you'll come out on top of this situation.
on Mar 31, 2006
Hire me.
on Mar 31, 2006
Trust me, if I had the credentials, i would want to work there myself.

I just did a presentation on GUI's thoughout history for school and will end up doing it agi n for my business workshop a few months from now. I sent a few (well, more than a few) people here already. Its just going to get bigger I guess for your guys as more poeple realize just what they can do wih their computer desktop and how they can intergrate online and offline with these technologies.
on Apr 02, 2006
My sympathy. It suprised me how many people didn't know what a beta was. I thought any gamer should've known that.


Haha, I totally agree. I read the GalCiv II forums very often, and its funny, but almost every time that Brad writes one of these interesting articles, I can pinpoint the exact forum topic that was the fodder for Brad's concerns. It may just be in my head, but there always seems to be a major forum topic that Brad comments in a day or two before he writes an article like this. Take the forum topic started in the past few days entitled "Do not download the second Beta 1.1!!!" in which the OP states there are a few problems he found and tells everyone else not to download the beta because it has problems. *rolls eyes* It would drive me mad! I really appreciate articles like this one written by Brad that give a little insight into how "users" like this affect the business of the company. Always an interesting read, Brad! Thanks!
on Apr 02, 2006
Yeah, good old mainstream. The further into it you get, the more users you get that put enormous support pressures on you. For a company like Stardock where two way communication between users and staff is large and visible, that can be incredibly dangerous.

To be honest I've been expecting this post for a few weeks. The masses of "I want THIS added to the game, why are they being evil and not changing the game for me?", "OMG I found a bug and reported it 5 seconds ago and it's not fixed, the game SUX", etc posts are quite depressing. Had the game come from Blizzard (not picking on them, they're just large and opaque) there'd have been none of that. With companies like Blizzard or Microsoft where NOBODY (Well there are exceptions. lol) expects to be able to actually speak with someone, mainstream users are far cheaper to support. Not expecting an instant reply, they might ask in an unofficial forum or even (GASP!) read the manual.

So what's the growing small guy to do? The important thing is to keep the power users happy and supplied with betas while keeping the support-heavy users at arm's length and away from betas. This is where 'level 1 support' can save far more money than they cost. Filter out the "is it turned on" type solutions that make it up 95% of mainstream support calls but more importantly give the power users a shortcut through to the existing systems as soon as they show themselves to be such.

Just make sure to get the balance right - keep the basic guy from getting confused and out of his depth without frustrating the genuinely knowledgable and helpful guy with "is it turned on? are you holding the mouse the right weay round?" questions.

Good luck!
on Apr 02, 2006
don't forget that you can also have *free* moderators (that is : they are not paid, they are just reliable fans willing to help)
most forums (even officials look at www.egosoft.com ) have moderators that are not here to answer questions but rather keep an eye on making the rules followed (such as : locking post answered in the faq or stickies, moving posts around in the right forum, merging threads talking about the same issue, deleting posts & threads such as 3 of the 4 posts of rosa here ...) and this allows the forum to continue working without too much problems.

anyway, I'm far from perfect but I wish you all the luck in the world, and to be able to handle all this new user base
keep up the good work
on Apr 03, 2006
don't forget that you can also have *free* moderators (that is : they are not paid, they are just reliable fans willing to help)

Well, I may be wrong but i think that all people with the rank of Governor have moderators right. And they aren't all stardock members.
on Apr 03, 2006

I think one of the things at work here with regards to support that hasn't been mentioned is that joe-user thinks a beta release is something very different from what we in the software world (well old school) think of. This is directly down to the effect of endless Google and Microsoft "betas", software that gets released into the wild for extended periods with no support and yet is treated as a fully released productioon ready product by droves of users. Unsupport "beta" game patches from the development houses (unsupported by distributors) have only served to reinforce this notion.

Thus, when Stardock posts a new "beta" what happens? Every man and his dog rush to download it because a beta to them is just another software release. Naturally they then recoil in horror when the thing crashes or a glaring issue crops up. Their expectations are wildly out of step with what Stardock is thinking when releasing a beta into the wider community. Yes there are a few disclaimers but who amongst these users reads such things? Isn't that just the usual legal jargon you get with Microsoft and co? Clearly not - but they're so used to blindly blazing past this stuff that they'll rarely stop to consider the implications.

Some of the suggestions above are right on the money in relation to drawing the power users into the fold and keeping them close, to provide you with that invaluable feedback you couldn't otherwise get with purely internal testing. Meantime you need to keep the masses a little more at arms length and keep them happy with all the frequent communications that we've come to expect and respect.

That said, there are frequent comments on these forums about the "feel" of these forums being off. I too never feel terribly comfortable with the SD forums and I can't quite put my finger on why or just what it is. I use a great many forums and there's just something not quite right here. Maybe you need a little more structure. Maybe it's because I can't seem to find my own posts (did they disappear?), and maybe it's just some basic navigation and cues that aren't there or rather aren't immediately obvious. But overall a cleaner more streamlined and intuitive forum should go a long way towards making users more inclined to look before asking and check faqs before asking the same old stuff time and time again.

Then you can more readily bury the beta releases a little more out of sight, and with some stronger warnings about just what the beta entails maybe you can reduce people's expectations a little and get back to the more measured feedback and dialog you're wanting.

Still, I have to profess that I've avoided the two beta releases because of the number of outright crash reports and apparent quirks of these new builds. I guess what bothers me the most is that through good intentions, a rush to deliver what the users want (demand?) and a desire to keep interest and sales high, there's a considerable risk that code stability will suffer. A buggy release could, in very short order, negate much of the amazing goodwill that's been built up in recent times and I for one would hate to see that happen.

From the outside looking in there are a surprising number of crashes and seemingly obvious issues being reported, although beta 2 looks a lot better. I appreciate that not having tried them i can't really comment with any authority on this, and I risk coming across as someone who reads all the moaning and takes it as gospel, but with your plans to release 1.1 next week I'm simply hoping that this upcoming release is getting the time it needs to properly cement the success that is GC2. Remember all these non-power-users that you're talking about will be a lot less tolerant of bugs and the need for incremental quick releases, so taking the time needed to ensure this build is watertight and polished is probably more important than ever.

Needless to say, the team at Stardock have my utmost respect and I applaud what you're doing and how you're doing it. You're setting new standards in customer interaction, customer engagement, openness and all around good humour and I cannot but be impressesd. But there are inevitably going to be some growing pains with the well deserved success of GC2 and the rest of your products.
on Apr 03, 2006
I've read all of Brad's post, but not all the comments.

Brad: Have an offer to sign up for beta-testing. Make the users go through a lengthy sign-up process (or something to weed out the chaff -- stardock central's ease of use kind of goes against that) and make them understand they are BETA testers (perhaps with an NDA).

The basic idea - which I'm sure brighter minds are working on - is to separate the ALPHA and BETA updates from the general public being able to just click & download them. You've made it too easy, I'm afraid to say.
on Apr 04, 2006
re #28: Horrible Idea... Keep beta testing the way it is!

I've no doubt that Stardock will transition. You guys are just so darn popular you need more folks, period. I am confident that you will grow and prosper.

The more popular you guys are the bigger chance that trolls and undisciplined kiddies will spawn in the forums. Cheers on getting some mods (moderators).

It's happend more times than we can count.. (You might be a galactic redneck if.... ) someone disses the AI for being a tard when they're playing easy level Someone claims the game is bugged to hell and after we get them to post their debug, we catch them with last year's drivers (or worse yet 2004! ) Someone claims they found a bug and its actually that the game was designed intentionally that way. Someone claims Stardock sucks, has the worst tech support ever, etc. when we KNOW that they are outright wrong about that, and they are expecting immediate response/miracles and/or being unreasonable. For the record, Stardock has wonderful, beautiful people that work in customer support. I wish Stardock owned more companies so I get that level of support elsewhere The latest one, (you might be a galactic redneck if) You tell people not to download the latest beta cause it has bugs. This one proves it to the world when you post it on the forums.
2 Pages1 2