White Shining Rock Logo
Close
White Shining Rock Logo
Close

Crunch Time & Compatibility Testing...

February 12, 2014 | Dukus | 103 Comments

Only six days left to release, and I've finally had a day that I'd call crunch time. I'm not sure one day counts as a crunch, but 17 hours in one day is a long haul. From the start of this project I didn't want to overwork myself, and I think I've accomplished that. One day of long hours in three years is acceptable.

However I'm glad I padded my release date with a few extra days – I haven't been stretched too thin, but I have been busy.

I spent the last few days putting together the release build and installers, and making sure they are compatible with all versions of Windows. And testing them. This is a painful process. Between wrong version numbers for the windows installer system, incorrect linkages to missing system DLLs, hardware failures of my development machine, and pressure to have everything done before this weekend, I'm feeling a little fried – but everything is set now.

I had been avoiding doing installer testing until I had final installers for the game. I know the game runs on Windows XP, Vista, 7, and 8 – testers have reported back that it works. But I hadn't tested fresh installs. I don't have enough machines to have all the OS's installed all the time – nor have I set up drives that can be mirrored to a clean install for repeated testing – so I didn't want to test fresh OS installs too many times. The OS install process just takes too much time.

So with a terabyte drive partitioned into chunks, I prepared to install XP, Vista, Windows 7, and Windows 8, and collected all the drivers needed on a USB stick.

The test procedure goes like this: First I'd install the OS and hardware drivers. Then I'd try to download the game and run the games two installers. One is for 32 bit systems and one is for 64 bit systems. If it failed I made sure that the windows installer responded with good error messages, like 'You need a newer service pack for the installer to work.' or 'This product isn't support for your processor type.'

Once the OS was properly upgraded and the installer worked, I'd test the game for a short period. Usually a video card driver would also need to be installed for performance reasons. Then I'd download and install Steam, download the game, and test the Steam version as well.

Then I'd test running the installers again, to make sure the repair/modify/remove dialog came up instead of the first time install options. I'd also make sure the game was properly listed in Add/Remove programs properly.

This whole process was fairly smooth but slow. But I did have to rebuild the installers with minor modifications a few times to get everything right.

Doing all this makes me want to give a shout out to the guys that test for PC compatibility for games day in and day out as their jobs. It's not glorious, and it's not fun - it's actually mind numbing. But its just something that needs to be done. Game developers – give some props to the people that test your games! If you've tested Banished and given me feedback – Thank you!!

Anyway, I think launch should go fairly smoothly, but I won't claim the game will run perfectly for all systems. Most all people that have already played the game don't have any issues, but I expect there will be a limited number of remaining startup issues and crash bugs that aren't yet discovered. But don't worry, I'll be here to try and fix them so you can enjoy playing. I'm going to be setting up support forums so that players can post crash dumps, save games, and images to help track down any remaining problems.

Only six days left!

Leave a Reply

Your email address will not be published.

103 comments on “Crunch Time & Compatibility Testing...”

  4. Compat testing is something i now outsource but used to do it all inhouse, so totally feel your pain.. if you need any help at all dont hesitate to ask 🙂

  6. Can't wait :).

    I hope you'll continue your dev blog also after the game comes out. It's very interesting and enlightening to read (even though I don't understand everything).

  7. I totally agree with Alicia. I am so very looking forward to this. Wish you all the best for release!

  8. Counting the days!
    'This product isn't support for your processor type.'
    Shouldn't it be supportED by? 🙂

  9. Can't wait longer! Have seen videos in you tube and only made the wait worse!

    Good timing, right after payday!

  11. To Álex ^

    Pretty much all VM software have very bad 3D acceleration, if they do, they still don't allow direct communication to the GPU, which then you can't play games.

  12. I used to build installers for Windows applications (e.g. Sound Forge, ACID, Vegas, etc) and I can reiterate that testing scenarios was mind numbing, and often lead to late nights spent debugging strange issues related to certain platforms in order to meet deadlines. One thing that our QA folks at the time utilized was Norton Ghost, with Ghost images for all major platform iterations we targeted. Of course, for just installer testing, I think virtual machines would work better... but we had to not only test installer issues, but also ensure that custom drivers, etc worked, which is harder or impossible to effectively verify on VMs.

  15. Man, you are the greatest for putting all this effort. I am so looking forward to the release day, I'll be playing the heck out of this for a long time, I can tell. I hope it's a great success and you have the resources to keep making fun games.

  19. If only you accepted beta tester, maybe with an higher price.

    Like planetary annhilation.. they let alpha tester buy the game for 90$ on steam.

    This would have helped you a lot, for testing and for rising more money from people that can spend them

  21. Hey Luke, if/when you have some time would it be possible to give the link to the forum more prominence on the site here? It's really easy to miss at present.

  22. I am SO ready to play, after watching all the episodes by all the "Let's Play" people on YouTube. I've logged many hours just enjoying their adventures.

    Nice tornado, by the way. Very believable!

  23. Regression Tester by trade here, can confirm - mind numbing!

    Keep up the good work! Credit Card is out and at the ready!

  24. Ahh, the testing days... Yea, they are long and mind numbing! I remember they days it would have a stack of clean install hard drives to test, took forever. My rule of thumb, was for every hour of programming, it took 4 hours of debugging. So, I can relate. But it will be well worth the reward and make money to boot! Keep up the good work.

  25. Why not hold back and release the game in March to make sure you have ID'd all the bugs.
    Just remember people, sometimes the act of waiting for something to come out is better than actually having the thing in your hands, so to speak.

  26. As a SQA professional... yep! Compatibility testing is painful, and far too few people even bother with it.

    This has bitten me personally many a time, since my 32b install of Windows 7 has given quite a few programs issues. I'm hoping someone in your beta group has a similar enough system that it'll be working for me, 'cause I'm REALLY eager to play this game!

    And don't worry - if it doesn't work, I'll be giving you detailed reports on what happened. Tear-stained reports, but detailed.

  27. As usual, great informative post Luke.

    If you've not already done so, please give a little consideration to a master bug list with numbers so people can see and comment on bugs without flooding the boards with identical posts.

  29. Great post...love the informative nature of your blogs...I'm so 'game bored' waiting for this release that i've fired up Knights and Merchants from GoG! 🙂

  30. @money You're making a lot of assumptions based on poor information. I'd suggest refraining from that in the future. The "early access" model does not fit all projects, and as can be seen on Steam lately, is ripe for abuse by unscrupulous developers who fail to disclose exactly what customers are buying into when they buy "early access". They choose to highlight incomplete or missing features, and minimize bugs or design problems with their "early access" games. The other issue is "early access" projects that build a lot of hype, get a lot of buyers, and then basically die.

    This doesn't even get into the fact that when you try to use "early access" as a way of getting beta testers and extra cash, it comes with a lot of strings attached that can hurt your project. Just as an example, you can't really fire a "beta tester" you get through "early access", also when people pay money for "early access" you get many of them have expectations about what the developer "owes" them for their pre-release investment into the game. IMO, "early access" is a bad trend in gaming that is being applied too broadly and should not be treated as a panacea for indie developers.

    I've seen a lot of people in the comments over the history of this project try to counsel Luke on how he should develop his business model for selling Banished. I'd say about 95% completely glosses over the downsides in the proposed business models and how it would impact the project. We aren't Luke, we don't know near enough details on his project or situation to be counseling him on what his business model *should* be. So let's not.

  32. @Sam
    Steam takes 30%
    If he puts a sale option on his website but still gives you a steam key, take that option, as he will get more money and also free advertising through your steam contacts 😉

  33. Windows XP. Given that Microsoft will no longer support XP after April, you probably don't need to do too much there as well. Just a thought.

    Congrats...the finish line is in sight. It's been a long haul for you, but well worth it. Your fans have awaited this day for a long, long time.

  35. That's great to hear.
    just wondering about the bug reporting (which sometimes can be a flood of double reports). I assume people will have to explain what was going on when the bug occurred, make sure it's reproducible, tell system specs, etc.

  39. Just wondering if you tested XP-64? It seems to be the OS everyone forgot about.

    Really looking forward to your game, keep up the good work.

  41. my main question would be have you tried multi GPU setups? as it's a pain having to disable SLi every time you want to play a certain game.

  42. Came here to add my 2 cents worth saying "Virtual Machines", and to say I will be buying this game.

    It's been amazing watching it grow, and you deserve a lot of credit. Also, it looks like a brilliant game I'll play for a long time to come.

    Well done!

  45. the problem with people who keep saying use virtual machines, you don't get the same driver checks as all the hardware is generic emulated hardware so it is heavily flawed when trying to test. more so virtual machines can never access the graphics card so they can only use a terrible emulated card with next to none of the features and abilities of the actual card in a PC.

  47. Love this game! CAn't wait for it to drop. Been watching the pre-release vids. Would love to see a menu chart to list the houses built, whether they're occupied or not, the number and ages of occupants, and an option to click on them to take you to the house for more detail info. Another point made by one of the pre-release guys, would be nice to see the ages of people dying, particularly of old age.

  48. Uhoh, I think I may be in more trouble than I realised.

    I hadn't watched any LP-vids of Banished until tonight. I came across Quill18's series of videos, and promptly spent a solid 4 hours enthralled by them while I should have been working!

    Watching the game in action is almost enough to bring tears of joy to my eyes. Not only is it the definitive successor to The Settlers 1 that I've been waiting to play for around 20 years, but it appears to be vastly better than my most beloved game of all time, in almost all regards.

    Hearing the sound of blacksmith's hammers ringing out from busy settlements just brings my childhood flooding back! 🙂

    Best of luck for the last 6 days, and for the inevitable post-release stress. May the launch go as smoothly as possible, and the required bug-fixing be minimal. You've certainly put in the work to ensure it will.

  49. For the folks saying 'use VMs' and 'use Automation' - that's all well and good, but you need to go through the process manually at least once to see the thing work the first time - other wise you don't know if it's your automation that's broken, or the actual system.

    Plus, setting up automation usually means you have to re-run the test many times while you debug the automation itself.

    Depending on the size of your team, the workload involved in automation may outweigh just doing it by hand.

    Automation is really useful for regression testing on larger products. i.e I made this change, did it break anything that used to work?

    I'm a software dev, and everyone in our team spend a lot of time on QA ensuring we've got the appropriate testing hooks set up.
    That's just for testing our application, which has only a command line interface, and we have absolute control of the environment it runs in.

    We still run into weird problems with Windows and other software we rely on that just randomly does something odd.

  50. I would also like to know which payment method get's you the most money.
    what percent does Steam take?, also if the game will be sold on your site can I use Paypal? other than online gift cards that's the only way I can buy things online.

    PS: thanks for spending the last 3 years designing and programming an amazing game.

  52. LOL, Gryphon. "tear stained reports". Made my day--er, night! 😀

    Truly looking forward to this with great anticipation!

  53. I'm ready, test me to see if I can take a download. lol Can't wait for game ,, thanks for information.

  56. you are a legend, thanx for keeping us informed in detail, regardless of how the launch ends up going it wont be negative because of your lack of trying.

  57. This looks like such an awesome game. Like Dan I've been watching the hours and hours of Qill18's vids. I mentioned the desire to have a chart menu for housing, I think the same can be said for crops - a listing of what crops/pastures you have, the most recent yield or number of animals, the number of workers, etc. Gives you an idea of what you've got in a quick/clear format instead of having to search all around once your town gets sizeable.

  58. Hi, Thanks for all your work. Like all in here im realy excited to play your Game. I would like to ask 2-3 Questions about MODing but i dont want to disturb. If you are willing to answer simply send me Mail. Thanks 🙂

  60. These blog updates are the best. Please keep them coming after the launch! I find them fascinating and well written. And yeah, I'm buying Day 1.

  61. It must be hard, but you make it sound easy! I'm really looking forward to playing this - I've seen a let's play on YouTube (Quile or something?) which has further whet my appetite 🙂

  64. Will their be a free demo out that is free to play and download?
    I want to try the game out before I make a purchase

  65. Your blog is entertaining to read, and I am wishing you all the best of luck. Giving the game to quill18 for testing is probably one of the smartest things in regard to marketing (even though I don't like the way he plays the game, I keep watching and wanting MORE!)that could be done.

    For payment, I will buy it off you, then add the steam key.

    As for additions to the game, modding is the way to go for small UI additions. Hope it will be great, and a steam-thread for this is awesome:)

  67. its the first time i can feel my hands reaching out for the game its nearly in my grasp after all the mayhem as quietened down how-about going on a world tour with quill 18 and akiss4luck who i be leave is coming to the uk in june lots of banished fans would love to see the great man in the flesh

  69. You sir are amazing. The dedication you have to your product and the customers is by far number one in my book. We certainly need more people like you who value their honesty and integrity. I have decided to buy several copies of your game for my friends as gifts to support you. It's a small contribution, but one I am happy to make.

  75. Hello !
    I never ever buy a game day one.
    Well.. Never say never...
    I will buy your game day one.

    Your game looks amazing, and I love reading about you work !
    4 days left... aaaaw maaaaaaaan... >_<

  76. omg so hype!
    been watching lots of gameplay videos on youtube and loving the game, even though it's not even out yet!

    will you be available for gameplay questions at any point? there's still a lot unknowns. for example: does a root give as much 'food' value as a pumpkin? or a piece of vennison? stuff like that.

  77. just the weekend to pass and were nearly there time to clear up my table top of rubbish clean the laptop clean the screen and put the sim city in the redundant pile

  78. You can upgrade a wooden house to a stone house- doesn't use as much wood for heating, also a dirt path can be upgraded to stone it's faster walking, that's all I know that you can actually upgrade.

  79. I've been hearing a lot of buzz about this game for months through the Simcity Guild that I'm a part of. To be honest the game never really interested me, but the closer it gets to release, the more I'm considering purchasing the game. If not only for the all around unique genre you've created within a city builder, but the fact you've done this all by yourself is truly inspirational and remarkable all on it's own. It's refreshing to see this and I hope release day goes smoothly for you, and everyone that purchases the game.

    Cheers to you sir.

  83. I like this game. It has a TON of potentail. But just remember this becareful. Steam....... changes games but. Also, would you open a donation link? I hav around 38 thousnad dollars left over from my real estate bisness, and i would lov to donate half of that.

  84. I honestly wish more game developers were like you. Constantly keeping in touch with your fanbase, and going above and beyond to create and maintain a fantastic game.

    Hopefully the bugs people encounter don't cause them to complain too much. It's just something that happens, and thats why there are updates and patches.

  86. on the release I know you said the the 18th, but can you possibly give us a rough time and timezone?

  89. @Kyosa the reason I ask is going by that it is about 8pm UK time, so I'm mainly wondering if that is the release time on all formats or just a placeholder on steam, as I've seen those countdowns be quite a way off.

  90. What would be the best way to buy this game??? I am thinking steam simply because, even though the DRM is a pain in the ass, I have changed my computer a few times over the years and files get moved around a lot. So that means that say I don't back it up and the computer crashes, I would lose access to the game entirely if it's a direct download? Unless you can re-download it with a code..

    I guess my two questions are, which way is it best to get

    How will non-steam download work?

  91. @Savion, by the sound of it even if you buy through the site you get a steam key, if this is the case you are better off buying through the site as I believe if you purchase through Steam that Valve takes about a 30% cut on the price of the game, so buying alternative methods could end up giving more money to help in his future projects.

  93. @Savion @Gav

    I read somewhere that bying thorught this site will give both DRM-free and steam key,so you will get the game on steam that way. The dev will probably get more money from it too.

    However, the game might not get to the "hot"-page on steam, which may not increase the numbers of buyers as much as if it where on the hot-page. I will buy through the website though.

  94. One thing I'm wondering is what TIME it will be available here on Tuesday...Steam is saying mid-day-ish, like around 2 p.m. Eastern...would love for it to be available here earlier! wooo!

  96. @pad11 thats what I'm wondering as well as it works out to about 8pm UK time so either way it's probably going to be around lunch time before anyone gets it if thats true.

  98. Congratulations on getting Banished out the door 😉

    Have been following your progress with this for the past 2 years and wish you a successfull release.

  100. It's been Feb 18 All Day! Haha it's hard being in this part of the world, in front of everyone and having to wait until the day after. Get all excited for nothing!
    Australia holds on...

  102. Hey dev

    My Banished will not start. I byed it on steam 22/02 2014. I had waited for this game in 1 month please help me 2 start it.

    From Sebastian

More Posts

Code Rot

April 17, 2022
1 2 3 … 47
Back to devlog
Back to devlog
© Copyright 2021 Shining Rock Software
Website Design & Branding by Carrboro Creative
menu-circlecross-circle linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram