Shit Just Got Real

Determinism

A fancy word that gets you a free drink at the nerd bar. Basically in the realm of programming we have a need for knowing the where and when of everything. Its kind of like having a Heisenberg compensator for real. If we know the where and when of things in a game it helps us keep all things in sync and makes the engineer a happy person. This is extremely helpful for network games, if I shoot you in the face on my machine, we want you to see the same fantastic blood spatter in your living room. If things are out of sync I might enjoy your exploding head but you are busy mining gold or some such silly online nonsense.

I give you my determinism manager, though not really a manager it sounds fancier. It gives you the ability to test determinism for just about anything your heart desires, all through the use of a few clever macros. Check that out. stay in sync. shoot in face. win for everyone.

///////////////////////////////////////////////////////////////////////////////
//
//
// Determinism Manager
// A Utility collection for testing system determinism
// Carson Fee 6/6/09
///////////////////////////////////////////////////////////////////////////////

// The macros provide an easy way to add and remove the testing code.
// They also enable us to get location information
#if defined(TEST_DETERMINISM)
#define DM_PRINT_SYNC(args) DeterminismManager::GetInstance().PrintArgs( __LINE__, __FILE__ , args ));
#define DM_DUMP_CALLSTACK() DumpCallStack(); // the engineer can add these to code before a variable is modified in a set
//function to see who all the modifiers are and to see who changes it just before the desync occurs
#define DM_HASH_TEST(pData) DeterminismManager::GetInstance().HashTest(pData, sizeof(pData), #pData, __LINE__, __FILE__);
#define DM_HASH_TEST_BLOCK(pData, nSize) DeterminismManager::GetInstance().HashTest(pData, nSize, #pData, __LINE__, __FILE__);
#else
#define DM_PRINT_SYNC(args)
#define DM_DUMP_CALLSTACK()
#define DM_HASH_TEST(pData)
#define DM_HASH_TEST_BLOCK(pData, nSize)
#endif


class DeterminismManager: public Singleton
{
// we may want two structures, one just for hashing and only throwing an assert
// and another like below which is much heavier, slower to generate, using more memory
// but is more helpful.
struct ComparisonData
{
uint32_t nFrame;
uint32_t nHash;
char aVariable[32];
char aString[128];
char aLocation[128];
};

// Start recording the hash values and any other state data we want to test against
void BeginRecord()
{
m_nCurrentFrame = 0;
m_eState = STATE_RECORDING;
}

// Set up indexes so we can start testing from the start of a frame and comparing
// the results generated from the replay with those generated from the frame
void BeginTest()
{
m_nCurrentFrame = 0;
m_nCurrentItem = 0;
m_eState = STATE_TEST;
}

// Stop testing/recording
void Stop()
{
m_eState = STATE_IDLE;
}

// increases the frame count, used to ensure we have the correct data a the correct time
void TickFrame()
{
m_nCurrentFrame++;
}

// During recording this function will store a copy of the string provided and generat a hash
// During testing the hashes will be compared. If their is a difference both strings will be displayed
// for comparison and an indication of failure will be given.
// If they are the same, the string will be displayed with an indiation of success
void PrintArgs(char* pVariable, char* pFile, char* pLine, char* pFormat, ...)
{
// we will put our string into buffer.
////////////////////////////////////////
// Formatting the string to store and hash
// it will be displayed on success and failure
const uint32_t BUFFER_SIZE = 128;
char buffer[BUFFER_SIZE] = {0};

va_list argList;
va_start( argList, pFormat );

int length = vsnprintf( buffer, BUFFER_SIZE-1, pFormat, argList );
/////////////////////////////////////////

if(m_eState == STATE_RECORDING)
{
AddHash(buffer, strlen(buffer), pVariable, pFile, pLine);
}
else if(m_eState == STATE_TEST)
{
// cache the current itme as compare hash has to
int nCurrentItem = m_nCurrentItem;
if(CompareHash(buffer, strlen(buffer))
{
// If the hashes match, just display the data
printf("OK:%s : %s\n", buffer, m_hashValues[nCurrentItem].aLocation);
}
else
{
// If the data does not match, so we are out of sync, display the original
// data and the recently generated data is incorrect
printf("OK :%s : %s\n", buffer, m_hashValues[nCurrentItem].aLocation);
printf("FAIL:%s : %s\n", m_hashValues[nCurrentItem].aString, m_hashValues[nCurrentItem].aLocation);
}
}
}


// This function hides whether the system is running in record or test mode
// In record mode it adds hashes the variables and records that data in a local table
// In test mode it hashes the same game data as during the record phase and compares this with
// the data generated during the record mode
void HashTest(void* pData, uint32_t nSize, char* pVariable, char* pFile, char* pLine)
{
if(m_eState == STATE_RECORDING)
{
AddHash(pData, nSize, pVariable, pFile, pLine);
}
else if(m_eState == STATE_TEST)
{
DumpCallStack(); // Code for this already exists
ASSERT_F(CompareHash(pData, nSize), ("The current state of the system has deviated from that recorded."));
}
}

// Generates a hash from the pointer provided and records location information
void AddHash(void* pData, uint32_t nSize, char* pVariable, char* pFile, char* pLine, char* pString=0)
{
ComparisonData& current = m_hashValues.Add();
current.nFrame = m_nCurrentFrame;
current.nHash = HashMem(pData, nSize);
// check length of strings , valid poitners etc
sprintf(current.location, "%s:%s", pFile, pLine);
sprintf(current.variable, "%s", pVariable);

// This is only used if we are using print args
if(pString)
{
sprintf(current.aString, "%s", pString);
}
}

// this function checks our recorded hashes and compares them to a hash of the current data
// This is only usd during Test mode
bool CompareHash(void pData, uint32_t nSize)
{
if(m_nCurrentItem < m_hashValues.Size()) // check we are still in a valid range of test items { ComparisonData& currentFrame = m_hashValues[m_nCurrentItem]; if(currentFrame.nFrame == m_nCurrentFrame) // confirm we are on the correct frame { if(currentFrame.nHash == HashMem(pData, nSize)) // does our recorded data match the current state of the recorded data { m_nCurrentItem++; return true; } } printf("VAR:%s, FILE:%s\n", currentFrame.variable, currentFrame.location); } return false; } enum eState { STATE_RECORDING = 0, STATE_IDLE, STATE_TEST; }; Array m_hashValues; // This should be something like ChunkedFileReplayBuffer
uint32_t m_nCurrentFrame;
uint32_t m_nCurrentItem;
eState m_eState;
};

///////////////////////////////////////////////////////////////////////////////
//
//
// Example use of Determinism manager
//
//
///////////////////////////////////////////////////////////////////////////////
// I'd like to be able to do this in responce to a tick event, but I think we will have issues with the
// ReplayManager using the PreTick during playback. This shouldbe examined.
// The solution may be to put this into the Logic::Poll function before the input and the system Tick has been performed.
MainLoop::Poll()
{
...
DeterminismManager::GetInstance().TickFrame();
...
}

// Here we are generating a hash of the 16 bytes of memory from the address of m_msgFlags onwards during recording and saving that in our manager
// On playback we will generate the hash again and compare it to the recorded one. If it does not match we have a determinism problem

Entity::HandleEvents()
{
...
if(msg.id == iMsgRunningTick)
{
HASH_TEST(&m_transform);
}
...
}

// We have to use the replay manager(or Juice) to ensure we have an EXACT copy of the actions done the first time are performed.
ReplayManager::GetInstance().Record()
{
DeterminismManager::BeginRecord();
}

// We have to use the replay manager(or Juice) to ensure we have an EXACT copy of the actions done the first time are performed.
ReplayManager::GetInstance().Playback()
{
DeterminismManager::BeginTest();
}

Why Artists hate Programmers

I have worked in games as both an artist and a programmer. My media of choice has been audio. I love sound, its taken me all over the world and brings me great satisfaction.
There has always been a great chasm between artists of any media and the nerds that work to support them. There are many reasons for this. While not always the case, programmers generally have spent time in university learning math and algorithm design. Artists generally have spent the same time smoking pot and throwing feces at a canvas. I'm a self taught programmer that went to a commercial school for audio. I smoked a lot of pot but also spent time learning the maths and watching Bob Ross paint on hangover Sunday.
This lesson is about how the nerds should approach the hippies so we can all get along.
The chasm is real, it is called "Toolset Chasm the Precipice of Sorrow" it was caused by years of book learning, passion, and 2 tribes that speak totally different languages.
We can break it down to wants and needs. Art needs tools, art wants the most transparent tools that allow them to produce beautiful things that make us laugh and cry. Programmers need tasks, programmers want to deliver the most elegant solutions that they can deliver in the shortest amount of time so they can move on to other tasks. They are equally passionate but the love is a different kind.
In the many job interviews I have had I describe myself as a person that can translate the unreasonable demands of the artists into reasonable tasks for the engineers. I'm really good at it "pats self on back". It is a rare skill.
Here is how programmers can join club awesome and roll like I
1. There is no such thing as programmer art. Programmer art is a text box, a spread sheet, a text file or some human (read super nerd) readable schema that represent the "content"
2. There is no such thing as content. It is art. Someone spent hours with some fancy gadgets producing the equivalent of their soul in digital form. Respect that, they aren't retards because they can't decompose a rotation matrix though they could be reards if they require IT every time they want to print said digital soul.
3. Stop saying No all of the time. Any tool is nothing more than a glorified text editor, they are not hard to make. There are still cases where No is still the appropriate response. Just not all of the time.
4. You have time. If an artist ask for a tool that uses a water gun to produce the assets for the game then go get a super soaker and build that shit. It is cool. The artists will make you a bacon apple pie or draw you your next space invader tattoo. What is sufficient is almost never what is most productive. Stop running to "There isn't enough time in the schedule". An unproductive artists will have you sitting updating facebook status until you need a catheter.
5. Its usually worth the risk. The other mantra of the programmer " At this point its too risky". See point 4.
6. Artists are shit typists. They don't want a text box that takes 3 floats to represent a position in space, they want an "Over there" orb of awesome.

Box of FAIL

ORB of AWESOME

7. A lot of the work is done for you already. Look at the creation tools the artists like to use. Most of them were bult by artists that are shit programmers. Steal their cool stuff and make it work better. Faders, color pickers, spline controls, waveform editors. There is a bounty of very nice controls that will have bacon on your plate in no time.

There are more rules but you are smart enough to figure them out. Start with making it beautiful, then make it work. I kmow its contrary to your very being but its what art needs.

Here's how artists can join club awesome and live with the hot ones:
1. Stop whining. The big bad programmer is not there to crush your dreams, he's there to build the machine that will get your life's work into the hands of the sweaty masses. Programmers are just as sensitive and reactive as you are, just for different reasons. Try not punching them in the face or writing 37 page emails about how the lack of an orb of awesome will completely halt your ability to work.
2. Teach the programmer. They are smart folks at a level equal to your talents. Show them how you do your stuff. They will not put you out of a job but they will have a better idea of the tools you really need to make the beautiful.
3. Learn from the programmer. Don't learn C++. You'll check in dangling pointers and break the build (Something I have done, twice). Learn a bit about how your stuff gets turned in to their stuff. Your talent is at a level equal to their smarts.
4. There are people that can do both. Usually they are called tech artists. If you don't have one, hire one today. They are thoroughly good. They are like tour guides, they know all of the cool spots in town and can translate with the locals.
5. You do not have time. Well you do just not unlimited time. Understand the schedule and build it with the programmers. Instead of needing 3 months to produce 10 different 5 meg cat ass textures. Build one good one and take it to a programmer, they can make something procedural that can reuse one cat ass texture to look like 10.
6. You are broke. You have a budget that is never enough to get all of the stuff you want. Welcome to the life of an artist. If you have to have all of those cat asses, see point 5.
7. Buy them beer. Programmers like beer. Share the beer with them. After a few pints they will be pulling out the pocket protector and will listen to your craziest ideas. Some of them may even make it past hangover Sunday and get in to the game.

Following these guidelines will begin to fill the chasm with love. A game team is a complete team of art and science. When there is harmony you will begin to see beautiful things take place. The nerds will talk to you. It will be like an episode of glee, but without the Journey cover tunes.

Both tribes are free to disagree with me. I accept insult in either language.

C. Artist, Programmer, Chasm of Sorrow filler.

Abstract with caution

Abstracts,

Interfaces, Factory Pattern these are fancy words that get thrown around by
programmers on a regular basis. The only
word I hear more is “Refactor” which makes me want to kick puppies but it is
part of the lingo.

An abstract class is essentially a base description. It would look something like so:

//AbstractFoo a base class for

stuff that likes to Bar()

class AbstractFoo

{

virtual void Bar(int a_nID) = 0;

};

Now the highbrow definitions of Abstract Classes and
Interface Classes says that an abstract has at least one pure virtual function
like Bar(int a_nID) = 0; the = 0 bit is
the thing that makes it pure virtual. I
weep if you don’t already know that. An
Interface class has no data members and fancy smanchy boys will prefix an
interface class with the letter “I” like IAmAnInterfaceClass, or
IICaptain. Abstracts can’t be
instantiated, we can’t ever have a new AbstractFoo(), its not allowed, no Foo
for You! You can instantiate interfaces
until your blue in the face, but a caveat is that since they have no data
members you should be aware of your allocations in the multitude of functions
that your interface contains. I had an
instance once where a debug interface was allocating 1 meg from the stack every
frame as a string buffer, this worked fine and was only debug so who cares? I
care. I had to port it to another
platform that was a little shy on the memories and proceeded to explode any
time it tried to allocate that handy meg of stack it did not have.

Abstracts are fantastically useful, the flagship of
polymorphism. Polymorphism to me has
always sounded like a fake word, like part of a magic spell Dr. Strange would
call out in battle. The idea behind the
abstract is that it describes a “thing” that you will use that performs common
functions. Its extensible because you
can swap out the “thing” with a different “thing” with relative ease provided
your replacement “thing” has the same functions.

Here’s a simple example.

Drinker.h

class Drinker : public AbstractFoo

{

virtual void Bar(int a_nID);

};

Drinker.cpp

void Drinker::Bar(int a_nID)

{

if ( a_nID == eBarsWeLike::Cheap )

{

//do drink

}

}

Alcoholic.h

class Alcoholic : public
AbstractFoo

{

virtual void Bar(int a_nID);

};

Alcoholic.cpp

void Alcoholic::Bar(int a_nID)

{

//do drink

}

See the differences?
The derived classes MUST implement the pure virtual function in the
Abstract lest they face the wrath of compiler errors and general whinging. The virtual keyword in the derived class is purely there to remind us that this function came from a virtual declaration
somewhere and also that classes derived from the derived class can override
should they choose to (but don’t I won’t like you if you do that). So we have two different “thing” classes that
both perform a function that is described in our Abstract. We use this in this way:

AbstractFoo* pFooBase = new Drinker();

pFooBase->Bar(1);

Sneaky huh? We
instantiated a new Drinker but we’re pointing to it with the Abstract
type. We can’t instantiate the abstract
but we can use pointers. The Polymorphism is really the compatibility between
the base and derived pointer, we can use one pointer of the base class to call
functions on many different types of derived objects.

AbstractFoo* pFooBase = new Alcoholic();

pFooBase->Bar(57);

We just changed the derived type to Alcoholic and called the
same function! The interchangeability is staggering! Its also very handy when we
have collections of several derived types, our collection would be base
AbstractFoo pointers and we simply call the function on each one:

AbstractFoo* BUKKIT[20]

For each AbstractFoo* pFoo in
BUKKIT

pFoo->Bar(1)

Its tasty, its made of win. Its something you already know right? Then welcome to club awesome, sit down and
relax. I have come to realize that this pattern is

really not that useful for certain circumstances. I work with Audio a lot, I work with engine
code a lot too. Every game engine I have
touched in recent years has employed this pattern for things like Audio,
Physics, and GUI. Stop it. Now.
Today. It really isn’t necessary. In the world of game development for audio
there are 3 libraries (yes there are more but they are really bad). I have had to switch these libraries exactly
3 times in the last 10 years. The core
functionality of each of these libraries is so fundamentally different that it
was a complete waste of time to add the extra layer of an Abstract. They are not simple cut and paste replacement
jobs. Same with Physics. 3 libraries, very different animals.

The lesson here is that an Abstract is cool,
but its like Fonzy cool, you like hanging out with him but you’ll punch him in
the face if he goes near your sister.
You need to pick your battles when an abstract presents itself. Is there enough common functionality? How alike are the various source chunks that
might fit the abstract. If you find
yourself adding a bunch of pure virtual functions to the abstract then you
probably don’t need one. Just jam that new
tech in there and giv’r.

C. : public AbstractDude

Strings Can Suck It slight return.

I'm phoning it in for this one. I wrote this a while ago at work for the youngins. It is still relevant months later!!!

I hate strings. I hate few things, hate is a strong hurtful word. I hate strings. –Sounds like a song lyric.

I digress, strings are horrible horrible things. They are human readable, but we don’t write code for humans to read, we write it for computers. They can convey a great deal of information, they can also confuse the hell out of things in the many contexts afforded in the English language (26 different meanings for the word “set” really!). Strings in Games is an oxymoron. The two do not belong together. Ever. Even if it’s a text based adventure game, which if it is you have a time machine and live in 1983 when text games were cool.

Why the hate Playa? What’s so bad about them sweet strings? I’ll tell you. As I said I hate them:

1. They take up a lot of run time memory. 1 Byte per character doesn’t seem like a lot but when you get in to the thousands of sentences it eats up the memories.
2. They fragment and pollute memory. They are variable length. This makes the cache gods weep.
3. Programmers abuse them. The number of times I have seen strings passed by value makes my eyes bleed.
4. You have to write all sorts of stupid conversion tricks. Remember your old job interview programming tests? AtoI or change case? Blah blah blah
5. They are like needles poking holes in your memory balloon. Pointer arithmetic, failed bounds checks, stomps! Corruption! Almost always because of a string.
6. They are slow. We get clever and add a bunch of “stuff” into them i.e. “give_finger_hand_right” this needs to be split up into its parts when a simple event id would do.
7. Everyone has their “own” awesome string class. It is not awesome. It is a string class. Ever see that Myth Busters episode where they polished animal poops into shiny spheres? String classes are like that but without the moustache dude.
8. We abuse them for weird type conversion. String streams are not a good solution for the absence of RTTI. That’s sooo 2002. If you really need reflection (you don’t) templates will work fine provided you don’t go the crazy with them (This will be my next week rant).
9. I didn’t even talk about Unicode. Unicode is like Satan’s string steroids. Unicode can internationally fuck off.

What’s it all mean Basil?

I hate them. Make them go away.

How?

Hash man, it’s all about hash. Now hashing isn’t a magic wand, it has not ended revolutionary wars to my knowledge but hash values are like beautiful flowers growing in the dirt that is strings.

A hash function is any well-defined procedure or mathematical function that converts a large, possibly variable-sized amount of data into a small datum. (from the Wiki land).

Here’s the basic idea:

“There once was a man from Nantucket who kept all of his strings in a bucket” -> MagicHashUnicornFunction() -> 4

All of that limerick converted to the number 4! Beautiful! How does that work? I don’t care, there are a million different hashing algorithms. You could use 64 bit hashes or 32 bit hashes but the generally idea is all of your huge polluting, variable sized, cache smashing, memory stomping, not cool, weird converting is condensed into a fixed length nice number that will fit nicely in a cache line and win you a frosty malt beverage from me at the pub.

We still need to have the string somewhere if it is required to display it on the screen. That unfortunately is unavoidable but it can sit on disc where our nice hash ultimately can index it and send it off to mister GPU and not hang about in our heaps or stack making the engine groan. I’m going to write a small test app that will use hashes as an index to an array of “proc-snippets “. My new made-up word. A proc snippet will be like a teletype ribbon (you’re old like me if you know what those are, I use to send love notes to one of the receptionists at Texas instruments when I was 5 over a teletype). We will use the proc-snippet to procedurally generate a sentence. Just out of curiosity as to how fast/slow it would be vs. disc access of said sentence.

Should you disagree with my hatred of the string please enlighten me with a book reference of your choice. I will hash the entire book and reply with the number 4.

2e5905a6

Tabs and Spaces can suck it

Coding Standards,

Love them or hate them they exist for a multitude of reasons. They are a paradox, both useful and useless. Created with the best intentions by a group of assholes. I am one of those assholes that has contributed to a code spec or two. The fundamental problem in the creation of a coding standard is that it has a multi-asshole paradox, like the highlander there can be only one. The coding standard has a committee of sphincters.

There's an old joke that states a camel is a horse designed by committee. This is what happens to a coding standard. Put a bunch of seasoned, jaded programmers in a room, say n number of programmers and you will get n opinions of how to do everything from painting a fence to shaving back hair. It all goes down hill from there. This group we'll call nProg will bicker over the smallest details and through some alchemy or battle of wills make choices that will affect all of the minions beneath them.

One thing that should happen in this group is at the first mention of the Code Complete book said mentioner shall be cock-punched and ejected from the room. Its not a bad book, its just so full of shit it makes me wretch at its very mention, it cleverly removes the ability for programmers to think for themselves. It is also 18 years old so it kind of suffers from age.

In order for nProg to be effective they need to work independently. You may be sitting back a bit thinking "what the fuck is this dude talking about?" What I'm talking about is breaking the coding standard into core elements and having one member to one element. Sub committees of one if you will. With each overlord having their respective sandbox they can impose their might on a specific element and enjoy the rewards such as how many spaces to indent brings.

This brings me to my point (finally). Here is my coding standard framework for the cloaked wizards that make the choices. they are not commandments but a selection of elements that you may chose to use or respond in comments that I'm a fucking loon.

Carse's Coding Standard For Hotness:

Files:

One class per pair of headers and source. Files should be named after the class contained within i.e. MyBadAssClass.hpp MyBadAssClass.cpp.

Use include guards or pragma once I don't care, effectively the same shit.

Header .hpp or .h go crazy

Source .cpp

Inline .inl

Assembly .asm

Functions:

Camel case Horse case brief case again I don't care just choose one style.

i.e.

GetCoolStuff

_GetOtherCoolStuff I personally hate the underscore and don't give a shit about its history

Scope:

Get a smart programmer to write this one. A horrifying number of programmers don't understand the differences in public, protected, and private. There also tends to be a loose understanding of public, private, and virtual inheritance.

In practice I avoid multiple inheritance but to each their own. I still screw these up from time to time hence get the smart nerd to write this one out.

Variables and Such:

Use Hungarian, Portuguese, Polish, Pig Latin, whatever floats your boat.

Member variables prefixed with m_

Member pointers prefixed with p_

Function argument prefixed with a_

Enums prefixed with an e

Get the cock punch gloves out again if people start using bitfields to save space for bools its retarded. However if this isn't your element then suck it and let the owner decide.

Singletons:

This will be another 50 page rant at some point. Do you need a singleton? If so build them correctly. If not then just make global static functions. Or make anonymous namespace functions.

Tabs or Spaces:

Truly these can fuck off. The tab key can fuck up formatting between editors. Big fat hairy deal. If you are not using Visual Studio to write code then you are either a meganerd that only uses vi or emacs or you make stuff in xcode for the iWorld. I don't care. The other 99.9999% uses Visual Studio. Stop using that university script kiddie shit and switch to a real IDE. Its one of the few things Microsoft did right. If it is still a huge deal, then set Visual Studio to insert spaces when you use the tab key. That way the professor on his 386 that might look at your code in his archeology project exhuming your computer and corpse will not have the alignment of text all fucked up.

Templates:

Templates are good, Templates are the Devil's language. Have a smart guy that can read more than 5 pages of Modern C++ Design without falling asleep write this one.

Its all common sense man, just give each grand master his own dojo and everything will be cool. There a a multitude of other pieces to a standard but this should get you going.

Thanks for reading, flame on.

C.