I like real world statistics. Here is a programming languages popularity comparison by Tiobe Software.
I'm going to start new fun project - Rubik's Cube. Eiffel Envision + .NET + NUnit + DirectX 9.0. Tools Envy ®.
Today, while downloading dx9sdk (225 MB), managed to create first test. Gotchas:There is also Agile Data Homepage by Scott W. Ambler. Lot of useful information, but maybe bit superfluous, especially for me right now. BTW, Scott W. Ambler's Agile Modelling mailing list was my last step before diving into XP.
Cover scan and other relevant changes in Books.
Downloaded Glowcode. Looked good - small and humble. Crashed every time when starting to profile. Uninstall.
DevPartner Profiler. Integrates (hides) into MS VS. Required to build my program with special profile data.
It worked for debug configuration, but failed with release build. Uninstall.
100 MB VTune. Last chance, better be good. And it really good. After I figured out linker "/fixed:no" option,
rebuilt application and profiled I got my bottleneck (as usual it wasn't that I thought of) and reduced it
to reasonable level. BTW, memory mapped file was approximately 3 times faster then fseek/fread in this case.
How it works. News stored in xml file. News filtering and presentation done via xsl. Cookie with last_read_id stored and retrieved by scripts and passed as parameter to xsl (because xsl can't do it itself).
Page has 3 modes of operation:Big XP projects with tens of developers are very rare. And so Martin's management experience is invaluable. Usual planning discussions on groups are quite simple and miss (as I see now) many important points. I definitely need some planning experience and then I'll read this book one more time.
Relevant changes in Books.3 weeks elapsed since I announced Rubik's Cube project. Time to ship something. Release 1 placed at the bottom of Code section. ebcl.dll (standard Eiffel runtime?) is 13.5 MB in size, that's why executable package is so big. You can run tests yourself from exe file using NUnit. Of course there is full source code written (rewritten from ### see my previous post) according to resume-quality and TDD principles. For all Eiffel discrepancies you should blame me (not Chet and nor Bertrand).
Stories that I tried, but failed to implement:Well, "more features to come" and I hope to unleash power of DBC in most appropriate language.
assert(resample([0 1 2 3 4], 2) == [0 2 4])
Test fails, code the simplest thing
function result = resample(input, new_step)
result = 0 : 2 : 4
function result = resample(input, new_step)
result = input(1) : new_step : input(length(input))
Instead of removing duplication we could use triangulation
assert(resample([2 3 4], 2) == [2 4], 'second first element')
assert(resample([0 1 2], 2) == [0 2], 'second last element')
assert(resample([0 1 2 3 4 5 6], 3) == [0 3 6], 'second new step')
function result = resample(input, new_step)
result = input(1) : new_step : input(length(input))
Tests pass, done.
From Kent's TDD book draft: "Triangulation feels funny to me. I only use it when I am completely unsure of how to refactor. If I can see how to eliminate duplication between code and tests and create the general solution, I just do it.Why would I need to write another test to give me permission to write what I probably could have written the first time?Here is my pros for triangulation and cons against first approach:
More than appropriate here a Writing as a Reflective Practitioner with Wisdom (beware unclosed header tag).
See also Top 10 tricks used by recruitment agencies!.
My projects really lack AT and I'm going to bring the customer closer to the development team. Teach her and help her create these testing scenarios.
Very interesting and easy to read paper on software economics Towards Measuring Software Value (you can hover mouse over indices to read notes) by Howard Baetjer.
"for reasons I will never understand, testers actually seem to enjoy this kind of thing" wrote Ron Jeffries about QA. You can see it for yourself in Test aerobics clip (9.5 MB) by James Whittaker.Two sites about non-standard job searching techniques: JobHuntersBible by Dick Bolles author of "What Color Is Your Parachute?" and Ask The Headhunter .
int signal_attenuation(){
const int maxThreshold = 42;
int result = free_space_attenuation();
if (result >= maxThreshold)
return maxThreshold;
result += obstacles_attenuation();
if (result >= maxThreshold)
return maxThreshold;
ENSURE(result <= maxThreshold);
return result;
}
Code above is untestable. OK, here goes second version.
typedef int (*IntFunction)();
int signal_attenuation(int maxThreshold, IntFunction free_space_attenuation, IntFunction obstacles_attenuation){
int result = free_space_attenuation();
if (result >= maxThreshold)
return maxThreshold;
result += obstacles_attenuation();
if (result >= maxThreshold)
return maxThreshold;
ENSURE(result <= maxThreshold);
return result;
}
Much better. We can even use boost::function.
int signal_attenuation(int maxThreshold,
boost::function<int> free_space_attenuation,
boost::function<int> obstacles_attenuation){
...
The problem is we don't need all this flexibility. Nobody will pass different parameters to our
function in runtime. Production code and tests already determined. So, the last version.
template <int maxThreshold, class FreeSpaceAttenuation, class ObstaclesAttenuation>
int signal_attenuation(){
int result = FreeSpaceAttenuation::free_space_attenuation();
if (result >= maxThreshold)
return maxThreshold;
result += ObstaclesAttenuation::obstacles_attenuation();
if (result >= maxThreshold)
return maxThreshold;
ENSURE(result <= maxThreshold);
return result;
}
class MockFreeSpaceAttenuation{
public:
static int free_space_attenuation() const{
return 1;
}
};
I'm glad to be able to express my design decisions in program code.
For detection I used _CrtMemCheckpoint and _CrtMemDifference functions from Debug version of C run-time libraries. First checkpoint placed before first test and second after the last. No need to change other code or project settings. It becomes a little trickier if you want to suppress Release build warnings or wrap this API in elegant class, but doable (although I failed to get memory leak size).
Yesterday installed Cincom Smalltalk. I already have a little experience with Squeak and now I'm reading Ivan Tomek's introduction. For me the different kinds of messages and their precedence were most confusing and this tutorial clears it in a moment.
Tomorrow I will promote acceptance tests first practice to direction.
One lesson for me - continuous communication is the key. Rare meetings are bad time to introduce new ideas, cause everybody have accumulated they own thoughts and want to talk more then listen.
"I don't always use TDD for throwaway code. When I don't, I almost always get in trouble and have a long debugging session. Slow learner." Ron Jeffries.
"If something (good) hurts, do it more often."
Using MS Visual Studio .NET and Visual Assist .NET for C++ projects, here is a list of functions (which use current text cursor position) that I have key bindings to:
| Copy line to clipboard | Handwritten macro |
| Paste line from clipboard below current line | Handwritten macro |
| Open line up/down to current (similar to Home/End + Enter) | Handwritten macro |
| Delete line | MSVS |
| Comment/uncomment block | MSVS |
| Increase/decrease indent | MSVS |
| Go to function/class definition | VA |
| Previous location (similar to back in IE) | VA |
| Build+run tests/run | MSVS |
| Open alternative file(.h from .cpp and .cpp from .h) | VA |
Other important features are auto completion and syntax highlighting (VA), tabs (similar to Netcaptor or MyIE) (MSVS), templates for class/test/function files creation (handwritten wizards).
In gVim (for Clipper/FoxPro projects) I additionally have "find same words and mark it" feature, just click the word to find holding Shift. Actually, it was the gVim that converted me. No need for toolbars and menus any more.I'm now using VisualWorks 7 a little for personal project. You know "Smalltalks' IDE are best". Well, I'll say you this: default configuration is ... questionable. Different windows (9 to the moment) clutter my taskbar, no user friendly toolbar, menu, key bindings customization. Good news: there are parcels for syntax highlighting, autocompletion, tabs. Today I hacked through to add "go to class in one key" feature. Something like "change your environment".
Programming patterns?
How it works:
1. There is no "tomorrow". Free time doesn't grow on trees.
2. More early is's fixed, more early you'll start to get benefits.
3. Quality work has strong positive feedback. And vice versa.
Which approach is better? I prefer the latter. Former looks more efficient, but to solve a problem is not enough. Do we need this solution at all? Do we need toolbar, what size of the window we want, is this a right test? We should think about it and interacting with an environment is not the best way of thinking for me. Fast feedback is important, but first define the goal.
What if we don't know what we want? Well, it's another question...I have a little experience using Windows Media Encoder 9 and this is how it works. I also recommend to download Window Media Player 9, it required for video editing and seems to have most recent codecs. I'm using Windows XP, but theoretically it should work under Windows 2000 as well. I'm using Genius Mic-01A microphone with frequency response from 100 Hz to 10 KHz and it sounds great. During recording my Athlon XP 1.667 GHz loaded up to 100% (70% on average).
Settings for WME9: Start from "Capture Screen" Wizard, capture "entire screen" and "audio", choose "high" quality, don't "begin capturing" and just click on "Finish". Go to "Properties", go to "Compression" tab and click "Edit", select "Quality VBR" video mode, go to "Quality-based" tab, select "Audio format" 16 kbps 22 kHz mono CBR, enter "Frame rate" 5 fps, go back to "General" tab, enter your favorite "Name" and "Export..." to unique profile file (otherwise some of your encoding settings will not be saved), press "OK" on the whole "Custom Encoding Settings" dialog. Now "Save" your session in file, next time just click this ".wme" file and you are ready to go!A picture is worth 1000 words, so check the bottom of Code section. There is little programming experience in that file, but you can check quality of picture and sound. You can also use such recordings for personal reflections <g>.
Without specification there is no correctness. Maybe this is the way to fight against "code first".
"... the programmers did not commit a bug. The team didn't do the right thing, it is true, but the problem starts with a missing Customer Test." Ron Jeffries.
What is your choice? Should we trust programmers or should we write more tests? Probably your don't have to make a choice. If you are a developer or a manager then trust, but when you are a customer then better check.When I first saw and tried CppUnit, the need to open test dialog somehow and click "Run" button every time I test was the reason I created my own UT framework that runs from command line and just displays assertions result in Windows MessageBox. I used the same approach creating UT framework for Clipper.
Then Eiffel and NUnit. I like NUnit's ability to collect tests from the program via attributes (no need to manage tests inside program). And we can start NUnit only once, later just clicking on "Run" button. NUnit also has assertEquals (to programmatically display nice descriptions like "Expected 1, but was 2.") and able to catch unhandled exceptions (to continue run other tests).Then VisualWorks and SUnit. Lack of description in assertions is my main objection (SUnit 3.1 has descriptions, but it displays them in common launcher window and so they are ineffective). Despite that SUnit has several neat features such as time elapsed from the last successful run, number of tests and yellow background while running tests.
In my recent Matlab UT framework I'm trying to combine these best practices, particularly creating a large set of equals assertions comparing matrices and signals with graphical descriptions. And now I'm back to the problem of one button testing. Maybe it's not a problem in VisualWorks where SUnit window is small and just one of the other 10, so we can constantly keep it near the corner of the screen. But using MS VisualStudio I can do my work having only one active window containing program code. I don't like to have to take a mouse, drag cursor to taskbar, select UT Gui window and click "Run" in it.Definitely there are programs to automate such tasks (I even wrote one myself), for example Tasker that I'm using right now. But I think it would be better if UT frameworks developers build some interfaces (COM, command line) into the program to facilitate true one key press operations.
Two most interesting things for me in this release are BOOST_CURRENT_FUNCTION macro similar to __FILE__ and __LINE__ (it works in vc7) and Filesystem Library.
WindowsXP controls in VW 7.1 looks little inappropriate as I use classic Windows scheme. Old windowSpec of my main window failed to load in v7.1, so I recreated it from scratch. To successfully deploy I included Toolbar resources in project (it used when you have a menu in program). Also I created separate image for deployment and I hope it will lighten the process (but definitely not to the level I have in C++). Testing using ForkedUI seems to cease to work too, but now I know that mixing acceptance testing with unit testing isn't a good idea anyway and I will try to use external tool such as TestComplete.
Removed link to Joel on Software (everybody knows about him without me and it's not so interesting now). Added link to just discovered Agile Articles (lot of resources divided by categories for easy search).
Added blogs section featuring Laurent Bossavit, Keith Ray and Brian Marick.Added agile-testing yahoo group (posters include Bret Pettichord, Lisa Crispin, Elisabeth Hendrickson, Cem Kaner from a testing world as well as many known leaders of agile community). Removed Boost Users group as of little value, particularly for the reader of the main Boost (developers) group.
Added Matlab newsgroup to the list. It's my champion newsgroup for high traffic and time I spend reading it (I'm newbie, Matlab is very big and there is absolutely no flame, just Q&A).
Transaction class>>newWithAmount: aQuantity from: fromAccount to: toAccount whenCharged: aTimepointOrDate
^self newWithAmount: aQuantity from: fromAccount to: toAccount whenCharged: aTimepointOrDate
creator: nil sources: Set new
newWithAmount: aQuantity from: fromAccount to: toAccount whenCharged: aTimepointOrDate
creator: aPostingRule sources: aSetOfEntries
^self new setAmount: aQuantity from: fromAccount to: toAccount whenCharged: aTimepointOrDate
creator: aPostingRule sources: aSetOfEntries
Transaction>>setAmount: aMoney from: aDebitAccount to: aCreditAccount whenCharged: aTimepointOrDate
creator: aPostingRule sources: aSetOfEntries
"private"
self require:
[aMoney isKindOf: Money.
aDebitAccount isKindOf: ServiceAccount.
aCreditAccount isKindOf: ServiceAccount.
(aTimepointOrDate isKindOf: Date) or: [aTimepointOrDate isKindOf: Timepoint].
(creator == nil) or: [creator isKindOf: PostingRule]].
self initialize.
self addEntry: (Entry new setAccount: aCreditAccount amount: aMoney charged: aTimepointOrDate).
self addEntry: (Entry new setAccount: aDebitAccount amount: aMoney negated charged: aTimepointOrDate).
creator := aPostingRule.
aSetOfEntries do: [:i| self sourcesAdd: i].
self checkInvariant.
Object>>require: aBooleanBlock
aBooleanBlock value ifFalse: [self error: 'Precondition Violation']
Transaction>>checkInvariant
|balance|
balance := entries inject: Quantity zero into: [:total :each | total := total + each amount].
self require: [balance = Quantity zero].
Look at DBC elements! When I started to use DBC in dynamically typed Matlab it was clear that
before checking function parameters value I should check their types (storing type in a variable name
like aQuantity for documentation is even worse than Hungarian notation).
And here it is, the approach known 6 years ago.
I hope nobody will call setAmount with integer money, cause [false. true.] value equals to true and precondition violation would not be noticed <g>.
Thinking about add and change reminds us about delete. Discussions about feature removal are rare and it may seem hard to do right, but in the light of this musings delete requires only to know how the program works. So delete = change - add. In other words, change = delete one thing and add another (on the other hand, delete often results in change of the other objects interconnections).
To minimize maintenance we can reduce number of change requests (using BDUF) or reduce cost of change (using solid software engineering practices). We can also try to transform change requests into the new feature requests, but following this principle while postpone the maintenance increase cost of change.You may ask what wrong in using good engineering, it certainly can reduce cost of change to the level of new feature addition. But it requires great skills and not everyone want to invest time in education. Unfortunately, I don't see how to avoid it, except of using (fragile and limited) big up front analysis.
How XP recommends to struggle with bad habits? Not. Code test and solution any (dirty) way down. Then recall code quality and refactor. These three steps supposed to be short term pleasant and despite that good in long term too.
You know all this jokes about "test addicted", "XP pusher" sometimes not so funny, for example when code 10 lines long too complex for me to comprehend, when I fear to refactor without tests, when I can't write code without user story and AT. Maybe it's my personal problem and not relevant to XP, but it seems like I'm very bad now in solving complex and abstract problems. Fortunately(?) I successfully simplify my current programming tasks, but type theory, lambda calculus, correctness proof, high level (UML) analysis are too complicated for me. When I read about projects like Sholis or when developers did not compile program for months and result yet has almost zero defects or when requirements thousands pages in size regularly changed and team handles it without claims of using XP I want to know how is it possible, I want to learn this techniques, but I can't.I want to be better developer, not simpler. Though there is a relationship.
From the shrink-wrap user point of view, looking at my programs I think the only product that was buggy enough for me is Win95, it was replaced by Win2000 in February 2000. All other programs were selected by the means such as price, compatibility and features.
From the in-house developer point of view, assuring quality (debugging and manual testing) took much of my time. Maybe it's my another personal problem <g>, but I can't throw raw code to my users. Instead I chose to study software engineering, which also supports program evolution.
Conclusion is surprisingly clear. One needs to be better developer to be more productive. My 2 main projects last 3 and 5 years, so I better be productive in a long run.(Other areas such as drivers, frameworks, languages probably have own peculiarities).
"Since there are often too many parameter combinations to test all possible scenarios, the tester must use some methodology for selecting a few combinations to test" The Combinatorial Design Approach to Automatic Test Generation.
Other essays by Michael Bolton. At least, read Zero Balances and Zero Responsibility, Installation Programs and Effective Beta Testing. No, you can't read only one <g>. Most entertaining is A Review of Error Messages.Duties of an application testing tool
1. Simulate user operations.Low level operations are easiest to implement. Sometimes they have user stories behind, e.g. "status bar should indicate object name under the mouse cursor" or "draw a red circle at the top of a graph".
When application GUI controls are standard OS controls, medium level also achievable. But not all text on the screen represented using controls -- labels and status bar information usually rendered as pixels.Application level control requires special developers work. It can be COM API, command line interface or exclusive extension.
Experience evaluating TestCompleteOne my project is a MSVC++ MFC app, other is in Matlab. All controls are native, so TC successfully drives them. There are extensions that allow to instrument MSVC++ program to be "open" for TC (call internal program functions from an external script), but I don't use them yet.
TC provide functions for screenshots capture, save/load and seek one image inside other (comparing parts of images less brittle to future changes).For test code you can use VBScript, C++ Script or Delphi Script (mini CLR <g>) inside TC. Outside you have access to all TC functions through COM. Special (.h) library allows you to cut'n'paste between C++ Script and real C++, e.g. Sys["ActiveWindow"]["WndCaption"] works equally well in the script and in the language. Very convenient.
Recording facility creates UI commands equivalent to user operations. Plain recording is exactly what makes GUI testings so brittle, but it's a quick start to familiarize yourself with the tool and beginning of the tests development. Help from the tool is the only way to deal with controls without title such as edit boxes, result looks like setEditText(p.inputForm["Window"]("MWEDIT4194304", "1", 9), "1000").VisualWorks applications are special. They don't use native controls, so you must use special tools. SilverMark's Test Mentor seems to be the one and only such a tool. It costs $3500, comparing to $350-$500 for TC. Anyway, evaluation version is always free <g>.
TM comes in parcels and intended to be used in development image. No support for testing deployed application, i.e. you can't test already deployed program and you can't test is everything included in deployed image during runtime packaging (I mean you can't test it automatically, manual testing is always with you)
TM's preferable tests development strategy is through UI recording. For example, you can add empty recording step inside test and TM will execute steps before planned recording (e.g. setup), then it will record your actions and then it will execute steps after recording (e.g. cleanup). So, you don't need to prepare application for recording manually. For verification you should select widget in the window using mouse, then TM will show list of parameters (specific for selected widget) and their current values. Selecting elements in this list automatically creates verification step. You can copy steps and scenarios, instead of recreating all from a scratch. Never before you could create lots of redundant data so easy <g>.Programming tests, main showstopper is that I failed to give UI time for message processing. Obviously, I can't write selectMenu: 'Add'. press: 'OK', because message for menu needs to be processed in UI subsystem -- then dialog with OK button will appear. Adding Processor yield or delayFor: 1000 after selectMenu doesn't help. In TM GUI shell's scenario this two messages form two steps. When TM executes them, it does insert appropriate pause. Unfortunately, I don't know how. Perhaps when running tests TM creates different thread for itself.
Thus, instead of plain old functions I'm stuck with a GUI containing steps with code blocks like
self varNamed: 'newsToAdd' put: (NewsItem fromArray: #(2 'April 1, 2003' 'text')).
CommonSuite runScenarioNamed: 'Add news'
TM practice is to create each assertion in different step. When unit testing, I often create several assertions in one function. Creating different step for each assertion is an additional work, but result more intention revealing. As more people work with AT and some of them may be non-programmers, it seems like a good approach.
GUI of TM itself quite ugly and buggy, while rich for running tests and analysing results. No problems during recording and playing scripts (I successfully reimplemented tests that were made using home-grown library). No descriptions in assertions. Little documentation and absence of discussions on web. All that is typical for VW.
So, it's a working tool. Disadvantages don't matter because there are no alternatives. Flashing windows of working ATs shine, especially if you don't know about lots of duplication ready to blow some day.All my ideas come from experience visiting other news resources. Most convenient scenario is when reader visit news page in exactly the same rate with it's updates (usually daily). If this is the case, news page should contain only new information. My style is different -- I can skip days, I can post several times per day and, most important, it's hard for me to post every day in the same time (but it's a good idea).
When reader visit news page more often than it updates, typical approach is to show her "yesterday news". Reading twice is a duplication that I want to refactor. Thus my Compact News (why almost nobody use it?).
When reader visit news page less often than it updates, typical approach is don't think about this case. Web top posters must die. Anti chronological order in news is like code before test. Thus again my Compact News where order is correct.
There is also first time visitors which interested in archives. Why typical archives are also in anti chronological order???
Here is my design. Main News page contains news for last month (32 days) in most simple (ac) form. Compact News page contains news for last two weeks (15 days). Archive page contains all news since the beginning. Permanent link to Archive is at the bottom of News page.
Perhaps I'll add more features later. Your ideas will be highly appreciated.
Basic idea is that given precondition function obliged to satisfy its postcondition. It makes interfaces very explicit and robust. Client knows what to prepare for function call and what a result will be. Supplier can rely on precondition that simplify its job to single responsibility.
For me DBC is the next big step after TDD (which in turn occurred 1.5 years ago). They have much in common: drive design before code, require some time practicing to comprehend, document code. But while TDD is deductive approach (by example), DBC is inductive (by specification). And as both specification and customer tests required for software product, DBC and TDD both help developer build correct solution.
See also this more detailed introduction to design by contract. But more importantly try it for yourself. Precondition checks are most valuable and don't require anything from the language. Postconditions harder to formulate and often TDD solution more simple.
Two examples which happened last week that I talked about at the beginning.1. For my Matlab project I wrote a function that snaps current mouse coordinates over graphic to data points. This function was thoroughly TDD tested with presumption that graphic's data (signal in time or spectrum) starts from 0. Later we added new type of graphic (autocorrelation function) that spans time interval from -T to T. It took some time to realize that cursor coordinates are now wrong and localize the problem. If I had explicitly written down precondition, I would immediately got error pointing to my (too restricted) function.
2. For my Smalltalk project I wrote a function for substring replacement. Presumption was that source string must contain substring to replace. Then I by mistake (wrong cut'n'past) passed to this function variable news instead of news_xml. Result was an internal VW library assertion and I spend some time to figure out that problem not in my function, but in caller. Again, if I had explicitly written down precondition it would been obvious immediately.
Today I killed the Bug which for two weeks terrorized our development team. Steps to reproduce problem were known and easy to follow (it occurred during program startup), but bug appeared only once in 10 to 20 runs when it knows we were hunting (and once in every 2-3 runs when we tried to work <g>).
As I discovered, the key was in mouse movement. If you move mouse over forming window, mouse move handler facing absence of (not yet created) principal objects wreak havoc. Actually, bug (unexpected axes object) was side effect of query gca command. gca returns current axes object and, if there is no axes yet, it creates one. The and clause is documented behaviour, but obviously I were not aware of it.
Lessons for me:How don't waste your time developing software? Ideal development would be typing all code in a row, then deliver application (or, even better than typing, just say it or draw it to computer). No, it's an ideal programming. Development also includes gathering and analysis of requirements, which in ideal world could just be written down as well.
Requirements simplicity comparing to bloated code comes only from omissions. Large part of requirements is in implicit form (doesn't "Clean up your room" extremely vague?). Using modern programming languages and complete requirements in English, programming is a barely syntax reformatting. And I think requirements is not complete without specification of how things should be done (having requirements/specification document we should be able to run our system in manual mode without computer). Software libraries reuse is exactly the reason for chasm between users/developers and programs. Of course reuse also has a lot of benefits and programming task is to bring it to project.
Debugging is a tool to better understand program. Testing to check our assumptions. Refactoring to evolve program. Modularity to manage complexity (calling lots of H2O molecules "an ocean" is also a modularity). It seems everything is important <g>. But novices do spend much more time on the same tasks comparing to experienced developers. So, difference is only in effectiveness. Time waste as inverse of productivity again has led us to importance of skills.
Software Testing and Internationalization book by Manfred Ratzmann and Clinton De Young is a modern view on (agile) application testing. Most things are familiar, but check out Testing by Documenting ("Does it really work the way I say it does?") and Interactive Unit Tests ("call a single function interactively" when number of test cases "goes into the hundreds").
I'll use this book as a reference to well known practices in which I have some doubts.
1. "Acceptance testing as Black Box tests". Not able to test everything, we must maximize effectiveness of chosen tests and White Box testing could point out important and dangerous boundary conditions. Maintaining tests, we must minimize their number and White Box testing is exactly the tool to remove duplication from tests (if some functionality implemented through common function, we don't need to test this functionality every test case).
White Box AT blurs the line between unit and acceptance testing, improves non developers' understanding of the product and enriches requirements/specification documents. Moving AT closer to developers removes duplication with UT and could be effectively combined with UI and interunit communications testing.
2. "Use a scripting language to produce tests". I use C++. Yes, it's my native language and all that, but rumors about disadvantages of statically typed languages are big exaggeration anyway. If testing is a software development, why shouldn't we use best development practices?
1. Our goal to is maximize business value of the product to customer for minimal cost and time. Code quality more important for development and maintenance (minimize cost and time). Business value is in working and specific requirements.
2. In XP, design, coding and testing are single activity. We can add requirements gathering and acceptance testing, but they still don't overlap - each activity helps in unique way (BTW, this is why we can't do "less XP", but we can do "more than XP").
3. Measuring time spent on each activity is easy, but without 1 and 2 it doesn't matter.
So, PSP is too narrow for me. It talks only about coding (TSP not better) and talks about work harder, not smarter - it doesn't introduce new practices. Well it talks about reviews, but from the paper I don't see what exactly does it mean - who, when and how perform this reviews? I hope it's not as bad as in Keith's and Ron's experience <g>.
PSP also strives for planning accuracy. According to Kari Hoijarvi: "It seems, that I'm probably the only one person in the world that finds both XP and PSP useful ... At least for me, the biggest measurable advantage of PSP is schedule prediction and tracking." I think, if it is important for you, just plan often.Nevertheless, I will try again to measure my time spent. I tried 1.5 years ago: for 1 month I manually recorded all activities. It requires very good reporting facility, I hope I'll create one this time. And I definitely need automatic tool, at least something like Activity Recorder. It's like profiling application - I want to see the hot spots.
But it's not the only source of fun in programming. Human beings have two source of "fun": pleasure and evolution. Evolution through creativity covered in the previous paragraph, the second part of the evolution is learning. Learning new things, there is no learning without new word.
Aren't there lot of new technologies in programming? The more experience I have and the more mature computer science is the more slowly new technologies come to me.Purpose of this reflection was to discover why I'm not so excited in development as most of the programmers (including myself in youth).
I easily give people second chance. It's very rare that there is absolutely no time or resources to correct a mistake. And once I have the final result this is the point to judge.
It's about acknowledging mistakes in public. Not very easy, right? Generally, there are two sources of failure: not enough effort and not enough knowledge. Both are quite unpleasant for the one who expected result from you. But effort and knowledge are not "mistake" as we know it. Mistake by definition is something small: typo, missed event handler, broken cup. Mistake by definition is something fixable. On the other hand, we have car crash or big loss of data witch are called "incompetence". This distinction is only by result, not by human activity.Thus acknowledging mistake isn't important - you will be judged by result. Acknowledging mistake sometimes presumes that you will learn from it. But, even if it true, it's "implementation detail" - there are lot of other ways to learn. Nobody care how you avoid making mistakes - just don't do them and everything will be OK.
Acknowledging mistake might have good psychological effect for the team: you signing to fix it and often free of charge, others looks smarter in contrast to you, you take off the blame from others. Making good things for the team pays back, but don't forget the cost.
Mistakes do have cost. Sometimes negligible, sometimes enormous. Try to control it.Holodeck product by Security Innovation. First of all read Software's Invisible Users by James A. Whittaker (do you remember his "Test aerobics" clip?).
Brilliant ideas! Interactions with OS severely underrated. James also talks about unintended functionality, which particularly dangerous in security applications (Testing for software security, scanned 15 MB article). And it's not only ideas, Holodeck performs all low level work to assist tester in exploring an application.
From the test first developer point if view, Holodeck can be used instead of mocking OS API. Ability to substitute any function in Win32 or .NET API allows unprecedented control over OS memory manager behavior, file access and network connections. For example, using samples and external interface from the trial Holodeck version, I created two small test applications in Managed C++: first denies file access to Notepad (so File Dialog doesn't show any disks, directories or files), second runs my own application limiting memory usage to 30 MB (my program gracefully managed first use case that required lot of memory and terminated on second). According to docs, I can substitute any system call with my own function or just return arbitrary error code. "Permanent" substitution could depend on input function parameters. "Dynamic" substitution might occur during program run, e.g. you can intercept interface just before particular test case.
While Holodeck is not free and first version GUI is not very useful and we yet can't handle more conventional human user interactions very well, this is the way to go. And using Holodeck API (which is always more preferable anyway), you can solve some kind of the problems right now.
Found via Eric Jarvi's blog.It's nice to think about general solution contrary to burdensome application specific trace messages, which cover only a small part of the picture. So, let's think.
Running as non-admin or Developer Lifestyle "...developing code as a non-privileged user and helping start a culture change on the Windows platform" seminal article by Keith Brown.
I always was a big (personal) security fan, reading and using appropriate stuff. And I always ran as a single most privileged user, but not anymore. Additional thanks to Craig Andera for popularizing these ideas.
Well, I'm not trying to sell security here. Rather I'm trying to bring out multi-user and limited privilege scenarios. Scenarios for which you should develop your applications. And, please, consider also scenario when user reinstalls OS and wants to preserve all settings <g>.
Try it one day. This is all-time experience.
It seems to be the best security blog available: joatBlog.
Top discovered link is a masterpiece called Caring for Your Introvert.
Top discovered security link is a Practical Approaches to Recovering Encrypted Digital Evidence. One clarification: quantum cryptography is to prevent eavesdropping at all.
Path: HKLM\Software\Microsoft\Windows\CurrentVersion\Explorer\BitBucket\NukeOnDelete
Result: ACCDENIED
Exception Management Architecture Guide. "This document discusses design and implementation guidelines for exception management systems that use .NET technologies. It focuses on the process of handling exceptions within .NET-connected applications in a highly maintainable and supportable manner."
What interesting is the exceptions in distributed applications and logging for maintainability. So, you can skip the basics (try, catch, finally) and start right from a Gathering information.
"TestComplete should be installed and used under the same account and this account should be set to administrator ... Perhaps, the situation will be changed in one of the future versions, but not anytime real soon." Bobby Edgar, AutomatedQA.
I don't want to work as admin and I don't have any alternative to TC. To think about...Found via Brad Wilson's blog.
Rubik's Cube intermediate release 2 is ready in Code section. It even solves sometime.
Besides Eiffel and Direct3D, I learned how not to optimize unrefactored code.Mike Clark presented Continuous Performance Testing With JUnitPerf article recently. I also did performance testing in several projects, I see a room for improvements in this generally unexplored area and I'd like to discuss it.
Simplest approach to optimize good working code is:start = getCurrentTime(); executeHotspot(); elapsed = getCurrentTime() - start; if elapsed > timeAllowed assertEquals(timeAllowed, elapsed);
Problems with profiler:
"... I was convinced that being able to separate what is important from noise was the main task of an engineer, and also that of a physicist. If you take into account all the factors that are involved into a certain real situation then calculations quickly become cumbersome and meaningless. Yes, meaningless, because very big formulas are very good at hiding what is really going on.
... you have to know what to leave out of your model, what to approximate. Maybe latter you will add a little something to the model and your results will get better; but now you already understand what is going on.
... abstraction is the key to being a better engineer. At least for me. Abstraction is a process by which what is important in some situation is separated from what is not important."
Radu Grigore, Notes 0001."As projects increase in sophistication, programmers often need a means to better reuse and customize existing component-based software. To achieve such a high level of code reuse, programmers typically employ a feature called generics. In Whidbey, the CLR will include a type-safe, high-performance, compile time-verified version of generics which differs slightly in syntax and greatly in implementation from templates as found in C++ and generics as proposed for the Java language. Generics allow developers to author, test, and deploy code once and reuse that code for a variety of different data types with negligible performance impact to applications. The CLR implementation of generics will also reduce code bloat when compared to other strongly-typed implementations, leading to more readable and maintainable source. Microsoft currently plans to support both the creation and consumption of generics in C#.
In addition to improving code reuse, Visual C# will eliminate some chaotic chores often associated with coding, such as implementing enumerator patterns. Iterators are constructs that dramatically simplify this task. Based on similar features in research languages such as CLU, Sather, and Icon, these constructs make it easy for types to declare how the foreach statement will iterate over their elements. Rather than having to create the classes and build the state machine for manually implementing the enumerator pattern, the C# compiler will translate iterator code into the appropriate classes automatically.
Anonymous methods are practical language constructs that will allow C# programmers in the Whidbey timeframe to create code blocks that can be encapsulated in a delegate and executed at a later time. They are based on a language concept called a lambda function and are similar to those found in Lisp and Python. Anonymous methods are defined dynamically at the point where they are used, rather then being pre-defined as a named method of a specific class. Anonymous methods make some types of operations more convenient, particularly when the method signature or body needs to be changed at run time."
Visual Studio + C# is already extremely (short term) productive environment. Now C# is moving towards long term projects, while VB.NET stays for simple tasks. Do not underestimate MS Language.Porting my custom wizards from VS 2002, I wrote a tutorial Custom wizards in MS Visual Studio 2003 for C++ projects.
I think it's enough for me to solve Rubik's Cube. Simple 4 steps look ahead + intelligent weight function just doesn't work. And I don't have any interest to implement complex algorithms required .
This experiment helped me a lot to try Eiffel language. I never thought those beautiful constructions are so hard to use. Lack of free functions and even static functions in classes; inconvenient loop with increment in a body, not in a header; embarrassing implementation inheritance, not composition; no enums; slow general functions like deep_clone and is_equal.
Nice features are, of course, native DBC; general functions like deep_clone; general persistence facilities. I have plans to finish OOSC and online future Eiffel book draft, then will come the final opinion.
So far, I've put release candidate executable in Code section. Now program has separate thread for processing, less delay between moves, ability to rotate cube planes from keyboard.
Hierarchies. Top-down decomposition. Divide and conquer. Does it make access to data easier?
Look at MSDN library, what do you use to read about some function? Contents? No, it's index. Look at file system, what do you use to open usual programs or files? File manager, Start menu? No, it's TypeAndRun or desktop/quick launch shortcuts (if it doesn't hurt your aesthetics). Look at writing, what do you use to save a file? File - Save? No, it's Ctrl+S.
The point is that all these intermediate layers are slow and unnatural. Natural way is 1 to 1 mapping (if I want to turn power off, I press Sleep button, I don't go to Start - Shutdown - ...).
How it relates to programming? Well, every programmer knows that most instant access is through global variables <g>. Strive for great names, prefer direct objects collaboration.
Next time you will structure, try to look further.
User Stories Applied for Agile Software Development book draft by Michael W. Cohn. This is work in progress and it covers user stories, requirements, planning, users, acceptance testing, XP and SCRUM. Everything is quite familiar for experienced XP reader. What particularly good in this book is a classification of users and separation of users from customers, managers and other not-a-real-users.
I wish I would find something about agile documentation. Besides the code, I see a need for:
1. Acceptance tests documentation for developers. Tests code is not enough, because:2. User guide for users and developers.
3. Short presentation (overview) for newcomers.
1 and 2 directed to preserve information, while 3 - to transfer it.
Michael also wrote an interesting small article on resource validation Configuration Bugs That Bite. Not that two solutions offered looks very appealing, but the point is to pay attention to this problem.
Programming made easier. Today, without any previous experience, I've wrote windows service, that maintains performance counter representing current HDD temperature. As access to performance counters from VBScript kind of complicated, I've additionally created separate WMI class and finally got it from vbs.
Documentation, frameworks and IDE from MS are programmers' best friends.
I would publish this app, if some settings were not hardcoded for my personal configuration. If anyone has interest, feel free to contact me.
Thinking about Proof vs Test (as in recent discussion in comp.object), sure, good proof guarantees better implementation than good tests, but there are other things that we are interested in software development - primarily design (managed complexity) and system evolution.
As of today, proofs are manual, while tests are automated. That makes tests more preferable for evolving systems. Tests (TDD) also help in design, making it more simple.
Now, what about other practices? What about DBC? DBC clearly is a design technique and automated almost as well as the tests. More interesting is that DBC also facilitates automated proof and simplifies implementation along with the tests.
Looking into the future, I see the needs for more design and more flexibility. Spreading DBC principles now is the right thing to do.
It also has some limitations - an applicability of OO in general, Eiffel in particular and year of publication (1997) in excuse.
All in all I highly recommend to read it.
Now I'm going to finish "Improving Web Application Security: Threats and Countermeasures" . Then I'll read ongoing work on future third edition of "Eiffel: The Language" by Bertrand Meyer. It's available here (username is "Talkitover", password is "etl3").
I've decided to list electronic versions of books in my Reading section. From the past reading I've included "Programming Applications for Microsoft Windows" Fourth Edition by Jeffrey Richter.
"Improving Web Application Security: Threats and Countermeasures" by Meier, Mackman, Vasireddy, Dunner, Escamilla and Murukan. Title is quite misleading, I would call it "Modern Windows Security: The Complete Guide". This book covers everything from TCP/IP to Web Services, from Registry to Isolated Storage, from ASP.NET to SQL Server, from code review to deployment. It uses theory (assets, vulnerabilities, threats, attacks and countermeasures) and practice (code examples, checklists, tools).
Why my current work doesn't have anything with security? <g> For those like me, I can recommend to read about code access security in .NET (security relations between assemblies) and DPAPI (W2K encryption API that uses the user's logon credential).
I've restricted my reading plans in Reading to a bare minimum and added my first XP book "XP Installed" to recommended part.
I found rubik solver in Amzi! Prolog + Logic Server samples. Core engine was written in Prolog by Dennis Merritt, but this product offers interfaces to other (modern) languages. So, my project got a new hope to be useful.
Release candidate 2 (see Code section) is now finally able to solve the cube, more fast and less memory hungry. The only thing left is the code cleanup. I'm also interesting how it works on the other computers - your feedback is welcome.
I'm also using NUnit 2.1 now, but there are hardly any useful features for me comparing to 2.0.
My Cube demo drops 300 frames for 60 seconds run at 100 fps. I thought it was .NET garbage collection.
There is a great tool CLR Profiler that you can read about and download from these 2 articles: Writing Faster Managed Code: Know What Things Cost by Jan Gray and Writing High-Performance Managed Applications : A Primer by Gregor Noriskin. It allows you to see life of every object in your application and it fast. When, for example, AQTime slows my demo by a factor of 10, CLR Profiler runs it almost in real-time.
CLR Profiler distributed in source code and you have to build it for yourself. I recommend to change its temporary directory from %WINDIR%\TEMP to something more appropriate (e.g. %TEMP%) and fix 2 bugs in ReadNewLog.cs (it fails when loading large logs with IndexOutOfBounds error).
You should run CLRProfiler.exe under Admin account. I recommend to turn off Calls tracing, otherwise log file size will be very large and processing will be slow. Most interesting analysis view is Time Line, which shows heap and work of garbage collector.
Well, GC is bad for applications like my demo. It could run anytime, stopping all other threads in (simple) application for 10 - 50 ms (1 - 5 frames). The only way to "control" it is System.GC class with Collect method, which doesn't even guarantee to compact heap (the most time consuming operation in GC cycle). But as GC runs only once in 5 seconds and compacts heap only once in 20 seconds (according to CLR Profiler) it can't drop more then 30 frames.
The real problem with performance turned out to be dependant on window size. With window of 400*400 pixels I had 300 frames dropped, increasing it to 600*600 I got 500 dropped frames. It doesn't depend on rendered scene complexity, so I think it's just raw video/memory/bus speed when copying frame buffers. I made a little tweak changing SwapEffect from Discard to Flip (it reduced drop count in large resolutions for 20%-50%).
Finally, I reduced default window size to 300*300 (100 dropped frames in 60 s) and considered it done.
BTW, a one way to measure frame rate in DirectX applications is Fraps utility. Although, Trace.WriteLine + DebugView is a way more flexible <g>.
When you Add Reference to your C# application in VS IDE, components in main list is not from the GAC. E.g. you may have DirectX 9 runtime installed and Managed DirectX libraries in the GAC, but you would not see it in VS.
To workaround this problem you can add key like this one to the registry:[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\AssemblyFolders\DirectX] @="F:\\WINDOWS\\Microsoft.NET\\Managed DirectX"(you can point to any directory and use any key name AssemblyFolders\My Key).
Then copy your dll files (like Microsoft.DirectX.DirectSound.dll) to this directory.
Reading on XP brings me less and less new information and I decided to stop. There are lot of other topics to master. My current interests are .NET, security, Eiffel and C++. I also very interested in functional languages and formal program verification, but I plan to study it later.
I feel myself obliged to update Links section.Refactoring, Extreme Programming and Test Driven Development are gone from my Yahoo! Groups. comp.soft-sys.matlab is gone from my Newsgroups - my Matlab project successfully finished 3 month ago.
Dale H. Emery added to Weblogs. Psychology is also one of my interest. The only reason I didn't link to Dale earlier - is that I though everybody knows about his page anyway.
Craig Andera, Raymond Chen, Brad Wilson, Chris Sells and Roy Osherove are added to .NET/Windows Weblogs. I posted links to them, so you should have your own opinion already.
Added to Security Weblogs: joatBlog - you know him, A Day In The Life Of An Information Security Officer - great investigation stories, two .NET security blogs by Ivan Medvedev and Greg Fee.
Management Weblogs section consists of two blogs by Johanna Rothman and one by Esther Derby
Two Yahoo! Groups added: Windows Technical: Off Topic - unofficial discussions in great company, Eiffel Software User list - Eiffel developers list.
3 lists from discuss.develop.com on .NET: ADVANCED-DOTNET, DOTNET-CLR and DOTNET-CX. I didn't try appropriate .NET newsgroups, but I think that signal/noise level in these lists is a way more better.
Two newsgroups added: comp.lang.eiffel and comp.software.testing (recent testing tools news).
Some blogs that I read, but didn't link: Martin Fowler's Bliki - it doesn't match my interests. Michael Feathers, Ron Jeffries and Christian Sepulveda started their blogs, but don't post much. And I recently discovered Pete Kruckenberg's Perfect World.
I don't read Troy Jessup's Security Blog, but it's the most beautiful one that I've seen.
// Generics: Stack<int> stack = new Stack<int>;
// Constrained genericity: public class Dictionary<KeyType, ValType> where KeyType : IComparable
// Implementation of iterator interface for class developers:
public class ThreeElementsList
{
public string foreach()
{
yield "microsoft";
yield "corporation";
yield "developer division";
}
}
public class List
{
internal object[] elements;
public object foreach()
{
foreach(object o in elements)
{
yield o;
}
}
}
// Anonymous methods:
public class MyForm
{
ListBox listBox;
TextBox textBox;
Button button;
public MyForm()
{
listBox = new ListBox(...);
textBox = new TextBox(...);
button = new Button(...);
button.Click += new EventHandler(sender, e)
{
listBox.Items.Add(textBox.Text);
};
}
}
See also little discussion on this document
and
Design and Implementation of Generics for the .NET Common Language Runtime paper
by Andrew Kennedy and Don Syme.
It's not as powerful as .NET CAS, but certainly useful.
function_call(value1,
value2)
You can also set eif-body-comment-indent to 1 (default is 0)
to shift comments off the code.
It's a great complement to OOSC. OOSC concentrates on design, while this book concentrates on programming. So, read first OOSC and then this book.
As it's a language reference, it's not always very entertaining. Parts of the book are still not finished and last update was 1 year ago. But you just have no other choice. You, as general Eiffel developer, can't get this information from other sources. And it's still has quality of Bertrand Meyer's work.
I am more comfortable with Eiffel now. When I've been reading OOSC it was mostly what I can't do in Eiffel, but ETL3 shows that Eiffel is quite practical and contemporary. Eiffel was one of the first languages with support of .NET and Eiffel Software still actively develop new tools and environments, offering free editions as well.
While Eiffel community is extremely small and Eiffel job market behaves accordingly, the language is quite unique/capable to justify its study. Meanwhile, I have 3 more weeks to finish and send my Cube demo to Eiffel Class Struggle 2003.
I recently received latest Bruce Schneier's book "Beyond fear" and Harry Beckwith's "Selling the invisible". It will be my current reading for now.
I tried to use VMware 4.0.5 to tests my Cube demo compatibility with Windows 2000 and 2003. Direct3D did not work under emulation. I mean, W2K virtual machine + DirectX 9.0b retail (redistributable) runtime + Managed DirectX demo + Reference DeviceType = Can't create Direct3D device.
It's not new. The news is that when I installed DirectX SDK Summer update on virtual machine it began to work. You don't have to install full SDK, Install DirectX Runtime + Install Additional Runtime Debug Files is enough.
Then I tried the same trick with the original (old) DirectX SDK and yes, it also works.
Cube demo v1.00 binaries (1 MB). It still requires .NET 1.1 and DirectX 9. New features: fullscreen mode, wireframe mode, cube maintains proportions when changing window size, option for black background, option for software rendering.
Source code will be released later.Jeffrey Richter's "Applied Microsoft .NET Framework Programming". Assemblies, reference and value types, constants, properties, events, enumerated types, attributes, delegates, exceptions, memory management, reflection, fundamental patterns (convertibility, equality, cloning, disposing).
This book focuses on IL level and also describes how many of C# language constructs maps to IL code. It's very useful for non C# developers.
No threads, no interoperability with unmanaged code, no remoting. Written style is good, but nothing special.
Overall, book is good. It's my first CLR book and I've learned a lot.
.NET lost several points in my eyes - lack of good design is obviously clear. Programming in C# without generics and multiple (implementation) inheritance requires to write a lot of duplicate code. Things will be better in Whidbey (C# 2.0), but then we will also have revisited MC++ (see also Ecma International Moves to Standardize C++ Binding for CLI) and, probably, Eiffel Envision 3.0 <g>.
Bruce Schneier's "Beyond fear". Security trade-offs, examples, analysis, statistics.
If you read Bruce, you will admire this new book. He doesn't repeat himself and writing is excellent.
If you don't know Schneier and security, well, each of his books is a right place to start.
Harry Beckwith's "Selling the invisible". Service marketing.
Services are different from products (we don't use them often, they are more hard to evaluate, we can make most of them yourself).
While focusing on marketing, the first thing this book suggest is to improve your service. Then it talks about naming, pricing and other visible things which make image of service in prospect mind.
The book is very small and reminds me "UML distilled" - lots of practical and smart suggestions, touching broad topics (all very relevant) and staying in focus, citing examples and other books.
Software development is service, especially when you use XP model. That was the reason I bought this book. But it gave me more - broader view on not obvious (but important) market aspects.
"Peopleware" (Second Edition) by Tom DeMarco and Timothy Lister.
Book for managers, not for me.