Wednesday, April 26, 2017

Tiips on writing your first book

I was asked, "What tips can you give to someone who wants to write his first book?"

Don’t worry about it, because the first book is just practice.

Maybe the first few books.

So stop worrying and write your practice book.

If your first book happens to be a success, that’s gravy. You can control whether you win or lose, but you can control whether or not you learn.

Get the book out there and gather feedback. Then use that feedback to improve your writing for the next book Repeat this process a few times and you’ll become a successful writer.

Do you recognize this iterative development process? You should, because writing a successful book is very much like writing a successful program.

In the meantime, if you'd like some help getting that first book finished, read


Saturday, April 15, 2017

Is it challenging to be a writer?

I'm often asked, "Is it challenging to be a writer?"

Knowing how many books and short items I've published, would you be surprised if I answered "No"?

Unlike many people, I would definitely answer "No." For me, NOT being a writer would be challenging.

Here’s another way to think about the “challenge” of writing:

Is it challenging to be a talker?

YES, for some people, but NO for most. You have something to say, so you open your mouth and words come out attempting to convey that “something.”

Now, the same frame for writing, at least as it is to me:

I have something to say, so I sit down by myself and words come out attempting to convey that “something.”

I suppose if I had nothing to convey, then writing would be a challenge. So far that hasn't happened.


If you're about to feel challenged, try reading Weinberg on Writing: the Fieldstone Method 

This book has given thousands of writers creative and useful “things to do” when they’re about to feel challenged.

For one thing, the book is full of exercises that can get you going. In the simplest of all the exercises, all you do is take out your writing instrument (computer, pen, pencil, typewriter, ...) and write the word, "The ". Now you've started writing, so all you need now is to continue. Try it!

Sunday, April 02, 2017

Complexity: Why We Need General Systems Thinking


It isn’t what we don’t know that gives us trouble, it’s what we know that ain’t so. - Will Rogers

The first step to knowledge is the confession of ignorance. We know far, far less about our world than most of us care to confess. Yet confess we must, for the evidences of our ignorance are beginning to mount, and their scale is too large to be ignored!

If it had been possible to photograph the earth from a satellite 150 or 200 years ago, one of the conspicuous features of the planet would have been a belt of green extending 10 degrees or more north and south of the Equator. This green zone was the wet evergreen tropical forest, more commonly known as the tropical rain forest. Two centuries ago it stretched almost unbroken over the lowlands of the humid Tropics of Central and South America, Africa, Southeast Asia and the islands of Indonesia.

... the tropical rain forest is one of the most ancient ecosystems ... it has existed continuously since the Cretaceous period, which ended more than 60 million years ago. Today, however, the rain forest, like most other natural ecosystems, is rapidly changing. ... It is likely that, by the end of this century very little will remain. - Karl Deutsch 

This account may be taken as typical of hundreds filling our books, journals, and newspapers. Will the change be for good or evil? Of that, we can say nothing—that is precisely the problem. The problem is not change itself, for change is ubiquitous. Neither is the problem in the man-made origin of the change, for it is in the nature of man to change his environment. Man’s reordering of the face of the globe will cease only when man himself ceases.

The ancient history of our planet is brimful of stories of those who have ceased to exist, and many of these stories carry the same plot: Those who live by the sword, die by the sword. The very source of success, when carried past a reasonable point, carries the poison of death. In man, success comes from the power that knowledge gives to alter the environment. The problem is to bring that power under control.

In ages past, the knowledge came very slowly, and one man in his life was not likely to see much change other than that wrought by nature. The controlled incorporation of arsenic into copper to make bronze took several thousand years to develop; the substitution of tin for the more dangerous arsenic took another thousand or two. In our modern age, laboratories turn out an alloy a day, or more, with properties made to order. The alloying of metals led to the rise and fall of civilizations, but the changes were too slow to be appreciated. A truer blade meant victory over the invaders, but changes were local and slow enough to be absorbed by a million tiny adjustments without destroying the species. With an alloy a day, we can no longer be sure.

Science and engineering have been the catalysts for the unprecedented speed and magnitude of change. The physicist shows us how to harness the power of the nucleus; the chemist shows us how to increase the quantity of our food; the geneticist shows us how to improve the quality of our children. But science and engineering have been unable to keep pace with the second-order effects produced by their first-order victories. The excess heat from the nuclear generator alters the spawning pattern of fish, and, before adjustments can be made, other species have produced irreversible changes in the ecology of the river and its borders. The pesticide eliminates one insect only to the advantage of others that may be worse, or the herbicide clears the rain forest for farming, but the resulting soil changes make the land less productive than it was before. And of what we are doing to our progeny, we still have only ghastly hints.

Some have said the general systems movement was born out of the failures of science, but it would be more accurate to say the general systems approach is needed because science has been such a success. Science and technology have colonized the planet, and nothing in our lives is untouched. In this changing, they have revealed a complexity with which they are not prepared to deal. The general systems movement has taken up the task of helping scientists unravel complexity, technologists to master it, and others to learn to live with it.

In this book, we begin the task of introducing general systems thinking to those audiences. Because general systems is a child of science, we shall start by examining science from a general systems point of view. Thus prepared, we shall try to give an overview of what the general systems approach is, in relation to science. Then we begin the task in earnest by devoting ourselves to many questions of observation and experiment in a much wider context. 

And then, having laboriously purged our minds and hearts of “things we know that ain’t so,” we shall be ready to map out our future general systems tasks, tasks whose elaboration lies beyond the scope of this small book.

[Thus begins the classic, An Introduction to General Systems Thinking]

Sunday, March 12, 2017

Classifying Errors

I received an email the other day from Giorgio Valoti in Italy, but when I wrote a response, it bounced back with "recipient unknown." It may have been a transient error, but it made me think that others besides Giorgio might be interested in discussing the issue of classifying errors, so I'll put my answer here and hope Giorgio will see it.

Here's the letter: 

Dear Mr. Weinberg,
My problem is that I’m looking for good way — maybe a standard, more likely a set of guidelines — to classify and put a some kind of label on software defects.

Is there such a thing? Does it even make sense trying to classify software defects?
And here's my reply:

Hello, Giorgio

It can certainly make sense to classify errors/defects, but there are many ways to classify, depending on what you're trying to accomplish. So, that's where you start, by answering "What's my purpose in classifying?"

For instance, here are a few ways my clients have classified errors, and for what purposes:


- cost to fix: to estimate future costs

- costs to customers: to estimate impact on product sales, or product market penetration

- place of origin in the development cycle: to decide where to concentrate quality efforts

- type of activity that led to the error: to improve the training of developers

- type of activity that led to detecting the error: to improve the training of testers

- number of places that had to be fixed to correct the error: to estimate the quality of the design

- and so on and on

I hope this helps ... and thanks for asking.

--------------end of letter-----------

As the letter says, there are numerous ways to classify errors, so I think my readers would love to read about some other ways from other readers. Care to comment?

Monday, February 20, 2017

How Long Can I Remain a [Ruby, Java, C++, Python, …]  Programmer?

Several respondents to an earlier post have asked me about the future prospects for workers in one programming language or another. Here's my best answer.

As others have said, "I can predict anything but the future." But also others have said that the only things we know about the future are what we know from the past. Therefore, you might get some idea of your future as a [Ruby …] programmer from the answers to a recent Quora question, "What were some jobs which existed 50 years ago but have largely disappeared today?"

It was great fun reading all these answers, many of which described jobs I held back then. I go back a bit more than 50 years, though, so I have a few more to add. Most obvious omission was the iceman. In the 1930s, we had an icebox (not a refrigerator, but an actual box that held a block of ice). The iceman’s horse-drawn wagon would come around regularly and be surrounded by us kids, hoping to get free shards of ice caused when he cut up little blocks to fit our iceboxes.

Another job only briefly mentioned was typesetting. I never held that job, but I was trained for manual typesetting for a semester in high school. At least I know where terms like upper-case and lower-case come from.

Someone also mentioned keypunch operator, a task (not a job) that was often done by prisoners who were literally chained to their machines. Who weren't mentioned, however, were key verifier operators. Not many people today have ever seen a verifier, let alone even know what one was.

Even before my time, there were jobs that disappeared, but which I read about—for instance, in a nineteenth century book about jobs for women. The final two chapters in the book were about a couple of sure-fire women’s jobs for the future (1900 was then the future).

First chapter was about teletype operators. The chapter “proved” that there was a great future for women because they could operate a telegraph key at least as fast as men (and the telephone had yet to be invented).

Second chapter was about picture tinters. There was, of course, no color photography, and it wasn’t really even conceived of. Women were supposedly much better at coloring photos because of their “artistic bent” and their more delicate hands. Though there are a few photo tinters still around today for special jobs, it’s not a career with a great future.

By the way, one future job for women that wasn't even mentioned in the book was typist (or amenuensis) in spite of the then recent exciting invention of the typewriter. Other sources explained that women would never be typists because everyone knew that women were not good with machines.

It's fun to think about these forgotten jobs, but they're also a source of important knowledge, or perhaps even wisdom. Job disappearance is not some new phenomenon caused by computers. It's always gone on through history. True, some jobs lasted a long time, so long that they were passed down from generation to generation, even becoming family names, such as Smith, Turner, Eisenhower, Baker, and Miller. (See, for example, <surnames.behindthename.com> for hundreds of examples)

Some of those jobs still exist, though often modified by new technology. Do you still recognize Fuller, Chandler, or Ackerman? And many others have largely disappeared, remaining only in some special niche, like photo tinters. Do you know anybody named Armbruster who still makes crossbows? Well, you probably know a few Coopers, but how many of them still make barrels?

No job is guaranteed. Nothing entitles you to hold the same job for life, let alone pass the job down to your children. So, for example, if you think of yourself as a "[Ruby…] Programmer," perhaps you'd better prepare yourself for future with a more general job description, such as "programmer" or "problem-solver."

In fact, there's a whole lot of people out there who think (or hope) the job of "programmer" will disappear one of these days. Some of them have been building apps since the time of COBOL that would "eliminate programmers." I've mocked these overblown efforts for half a century, but history has tried to teach me to be a bit more humble. Whether or not they succeed in your lifetime, you might want to hedge your bets and keep learning additional skills. Perhaps in your lifetime we'll still need problem-solvers and leaders long after we've forgotten the need for Chamberlains and Stringers.

Additional reading: 

Monday, February 13, 2017

Should I learn C++ or Python?

When I first saw this question on Quora, there were already 47 answers, pretty much all of them wrong. But the number of different answers tells you something: choice of programming language is more of a religious question than a technical one. The fact is that if you want to be a professional programmer, you should learn both—and at the same time.

When we teach programming, we always teach at least two languages at the same time, in parallel. Assignments must be done in both (or more) languages, submitted along with a short essay on why the solutions are different, and why the same. That’s the way to develop some wisdom and maturity in the coding part of your professional work.

Some of the respondents asserted that programming languages are tools. If that’s an appropriate metaphor, then how would you answer this question of a wannabe carpenter:

"Should I learn saws or screwdrivers?"

Do you think someone could be a top-flight carpenter knowing only one?

So, stay out of this quasi-religious controversy, which can never be settled. Instead, spend your valuable time learning as many different programming languages as possible, at least 5 or 6. You won’t necessarily use all of them, but knowing their different approaches will put you far above those dullards who say:

“I only know Language X, but it I still think it’s the best language in the world.”




Sunday, February 05, 2017

Fuzz Testing and Fuzz History

In 2016 I added a paragraph to the Wikipedia page on "fuzz testing." Later, the paragraph was edited out because it "lacked reference." The editor, however, suggested that I blog the paragraph and then use the blog as a reference, so the paragraph could be included. So, here's the paragraph:

(Personal recollection from Gerald M. Weinberg) We didn't call it fuzzing back in the 1950s, but it was our standard practice to test programs by inputting decks of punch cards taken from the trash. We also used decks of random number punch cards. We weren't networked in those days, so we weren't much worried about security, but our random/trash decks often turned up undesirable behavior. Every programmer I knew (and there weren't many of us back then, so I knew a great proportion of them) used the trash-deck technique.

The subject of software testing has many myths and distortions. This story of fuzz testing has several morals:

1. This type of testing was so common that it had no name. Apparently, it was giving the name "fuzz testing" around 1988, and the namers were thus given credit in the Wikipedia article for "inventing" the technique.

2. This is just one example of how "history" is created after the fact by human beings, and what they write becomes "facts." That's why I believe there are no such things as "facts"—not in the sense of "truths."

3. In any case, this is one example of why we ought to be wary of labeling "inventors" of various techniques and technologies. For instance, Gutenberg is often labeled the "inventor" of moveable type, though moveable type existed and was widely used long before Gutenberg. Gutenberg used this idea in ways that hadn't been employed before. That was his "invention," and a worthy one it was, but if we're to understand the way technology develops, we have to be more precise in our definition of what was invented and by whom.

Finally, I have no idea who "invented" fuzz testing. It certainly wasn't me.

NOTE: If someone would like to update the fuzz testing article on Wikipedia, they're welcome to reference this blog post.

Wednesday, January 25, 2017

What is the right reason to leave a job?

As a consultant, I frequently leave jobs. I also help many people decide whether or not to leave their jobs. I have learned there is no one "right" reason for leaving, but I've accumulated a list of many "good" reasons for leaving. I'll give some examples:

In my career, I have left jobs when 

- the job I was hired to do was finished.

- the job I was hired to do could not be finished.

- the job I was hired to do would be finished just fine without me.

- I was not able to do the job I was hired to do.

- the job I was hired to do wasn't worth doing.

- I was no longer learning new things (that's my most frequent reason for leaving)

- they told me that my pay was going to be "temporarily" delayed

- they asked me to do something illegal or unethical

You may notice that I never leave just because someone is going to pay me more money. If I was hired on to do a job, I feel committed to see that the job is finished, or going to be finished, or will never be finished. Only when my commitment is fulfilled am I ready to move on to bigger things. I don't think it's a good idea to leave behind me a trail of broken commitments.

Another good reason for leaving is not one I've experienced yet, but it's when they ask you to do something dangerous to your life or health. Very few jobs are worth dying for.


And here's a useful principle when leaving: If possible, don't quit until you have the next job set up. Why? Because it's much easier to get a new job when you already have a job. Employers tend to be suspicious of unemployed people.

Wednesday, January 11, 2017

Foreword and Introduction to ERRORS book

Foreword


Ever since this book came out, people have been asking me how I came to write on such an unusual topic. I've pondered their question and decided to add this foreword as an answer.

As far as I can remember, I've always been interested in errors. I was a smart kid, but didn't understand why I  made mistakes. And why other people made more.

I yearned to understand how the brain, my brain, worked, so I studied everything I could find about brains. And then I heard about computers.

Way back then, computers were called "Giant Brains." Edmund Berkeley wrote a book by that title, which I read voraciously.

Those giant brains were "machines that think" and "didn't make errors." Neither turned out to be true, but back then, I believed them. I knew right away, deep down—at age eleven—that I would spend my life with computers.

Much later, I learned that computers didn't make many errors, but their programs sure did.

I realized when I worked on this book that it more or less summarizes my life's work, trying to understand all about errors. That's where it all started.

I think I was upset when I finally figured out that I wasn't going to find a way to perfectly eliminate all errors, but I got over it. How? I think it was my training in physics, where I learned that perfection simply violates the laws of thermodynamics.

Then I was upset when I realized that when a computer program had a fault, the machine could turn out errors millions of times faster than any human or group of humans.

I could actually program a machine to make more errors in a day than all human beings had made in the last 10,000 years. Not many people seemed to understand the consequences of this fact, so I decided to write this book as my contribution to a more perfect world.

Not perfect, of course, but more perfect. I hope it helps.


Introduction


For more than a half-century, I’ve written about errors: what they are, their importance, how we think about them, our attempts to prevent them, and how we deal with them when those attempts fail. People tell me how helpful some of these writings have been, so I felt it would be useful to make them more widely known. Unfortunately, the half-century has left them scattered among several dozen books, so I decided to consolidate some of the more helpful ones in this book.

I’m going to start, though, where it all started, with my first book where Herb Leeds and I made our first public mention of error. Back in those days, Herb and I both worked for IBM. As employees we were not allowed to write about computers making mistakes, but we knew how important the subject was. So, we wrote our book and didn’t ask IBM’s permission.

Computer errors are far more important today than they were back in 1960, but many of the issues haven’t changed. That’s why I’m introducing this book with some historical perspective: reprinting some of that old text about errors along with some notes with the perspective of more than half a century.

1960’s Forbidden Mention of Errors
From: CHAPTER 10
Leeds and Weinberg,
Computer Programming Fundamentals PROGRAM TESTING
When we approach the subject of program testing, we might almost conclude the whole subject immediately with the anecdote about the mathematics professor who, when asked to look at a student’s problem, replied, “If you haven’t made any mistakes, you have the right answer.” He was, of course, being only slightly facetious. We have already stressed this philosophy in programming, where the major problem is knowing when a program is “right.”

In order to be sure that a program is right, a simple and systematic approach is undoubtedly best. However, no approach can assure correctness without adequate testing for verification. We smile when we read the professor’s reply because we know that human beings seldom know immediately when they have made errors—although we know they will at some time make them. The programmer must not have the view that, because he cannot think of any error, there must not be one. On the contrary, extreme skepticism is the only proper attitude. Obviously, if we can recognize an error, it ceases to be an error.

If we had to rely on our own judgment as to the correctness of our programs, we would be in a difficult position. Fortunately the computer usually provides the proof of the pudding. It is such a proper combination of programmer and computer that will ultimately determine the means of judging the program. We hope to provide some insight into the proper mixture of these ingredients. An immediate problem that we must cope with is the somewhat disheartening fact that, even after carefully eliminating clerical errors, experienced programmers will still make an average of approximately one error for every thirty instructions written.

We make errors quite regularly
This statement is still true after half a century—unless it’s actually worse nowadays. (I have some data from Capers Jones suggesting one error in fewer than ten instructions may be typical for very large, complex projects.) It will probably be true after ten centuries, unless by then we’ve made substantial modifications to the human brain. It’s a characteristic of humans would have been true a hundred centuries ago—if we’d had computers then.

1960’s Cost of errors
These errors range from minor misunderstandings of instructions to major errors of logic or problem interpretation. Strangely enough, the trivial errors often lead to spectacular results, while the major errors initially are usually the most difficult to detect.

“Trivial” errors can have great consequences
We knew about large errors way back then, but I suspect we didn’t imagine just how much errors could cost. For examples of some billion dollar errors along with explanations, read the chapter “Some Very Expensive Software Errors.”

Back to 1960 again
Of course, it is possible to write a program without errors, but this fact does not obviate the need for testing. Whether or not a program is working is a matter not to be decided by intuition. Quite often it is obvious when a program is not working. However, situations have occurred where a program which has been apparently successful for years has been exposed as erroneous in some part of its operation.

Errors can escape detection for years
With the wisdom of time, we now have quite specific examples of errors lurking in the background for thirty years or more. For example, read the chapter on “predicting the number of errors.”

How was it tested in 1960
Consequently, when we use a program, we want to know how it was tested in order to give us confidence in—or warning about—its applicability. Woe unto the programmer with “beginner’s luck” whose first program happens to have no errors. If he takes success in the wrong way, many rude shocks may be needed to jar his unfounded confidence into the shape of proper skepticism.

Many people are discouraged by what to them seems the inordinate amount of effort spent on program testing. They rightly indicate that a human being can often be trained to do a job much more easily than a computer can be programmed to do it. The rebuttal to this observation may be one or more of the following statements:
  1. All problems are not suitable for computers. (We must never forget this one.)
  2. The computer, once properly programmed, will give a higher level of performance, if, indeed,
    the problem is suited to a computer approach.
  3. All the human errors are removed from the system in advance, instead of distributing them
    throughout the work like bits of shell in a nutcake, In such instances, unfortunately, the human errors will not necessarily repeat in identical manner. Thus, anticipating and catching such errors may be exceedingly difficult. Often in these eases the tendency is to overcompensate for such errors, resulting in expense and time loss.
  4. The computer is often doing a different job than the man is doing, for there is a tendency– usually a good one—to enlarge the scope of a problem at the same time it is first programmed for a computer. People are often tempted to “compare apples with houses” in this case.
  5. The computer is probably a more steadfast employee, whereas human beings tend to move on to other responsibilities and must be replaced by other human beings who must, in turn, be trained.
In other words, if a job is worth doing, it is worth doing right.

Sometimes the error is creating a program at all.
Unfortunately, the cost of developing, supporting, and maintaining a program frequently exceeds the value it produces. In any case, no amount of fixing small program errors can eliminate the big error of writing the program in the first place. For examples and explanations, read the chapter on “it shouldn’t even be done.”

The full process, 1960
If a job is a computer job, it should be handled as such without hesitation. Of course, we are obligated to include the cost of programming and testing in any justification of a new computer application. Furthermore we must not be tempted to cut costs at the end by skimping on the testing effort. An incorrect program is indeed worth less than no program at all because the false conclusions it may inspire can lead to many expensive errors.

We must not confuse cost and value.
Even after all this time, some managers still believe they can get away with skimping on the testing effort. For examples and explanations, read the section on “What Do Errors Cost?”

Coding is not the end, even in 1960
A greater danger than false economy is ennui. Sometimes a programmer, upon finishing the coding phase of a problem, feels that all the interesting work is done. He yearns to move on to the next problem.

Programs can become erroneous without changing a bit.
You may have noticed the consistent use of “he” and “his” in this quoted passage from an ancient book. These days, this would be identified as “sexist writing,” but it wasn’t called “sexist” way back then. This is an example of how something that wasn’t an error in the past becomes an error with changing culture, changing language, changing hardware, or perhaps new laws. We don’t have to do anything to make an error, but we have to do a whole lot not to make an error.

We keep learning, but is it enough?
Thus as soon as the program looks correct—or, rather, does not look incorrect—he convinces himself it is finished and abandons it. Programmers at this time are much more fickle than young lovers.
Such actions are, of course, foolish. In the first place, we cannot so easily abandon our programs and relieve ourselves of further obligation to them. It is very possible under such circumstances that in the middle of a new problem we shall be called upon to finish our previous shoddy work—which will then seem even more dry and dull, as well as being much less familiar. Such unfamiliarity is no small problem. Much grief can occur before the programmer regains the level of thought activity he achieved in originally writing the program. We have emphasized flow diagramming and its most important assistance to understanding a program but no flow diagram guarantees easy reading of a program. The proper flow diagram does guarantee the correct logical guide through the program and a shorter path to correct understanding.

It is amazing how one goes about developing a coding structure. Often the programmer will review his coding with astonishment. He will ask incredulously, “How was it possible for me to construct this coding logic? I never could have developed this logic initially.” This statement is well-founded. It is a rare case where the programmer can immediately develop the final logical construction. Normally programming is a series of attempts, of two steps forward and one step backward. As experience is gained in understanding the problem and applying techniques—as the programmer becomes more immersed in the program’s intricacies—his logic improves. We could almost relate this logical building to a pyramid. In testing out the problem we must climb the same pyramid as in coding. In this case, however, we must take care to root out all misconstructed blocks, being careful not to lose our footing on the slippery sides. Thus, if we are really bored with a problem, the smartest approach is to finish it as correctly as possible so we shall never see it again.

In the second place, the testing of a program, properly approached, is by far the most intriguing part of programming. Truly the mettle of the programmer is tested along with the program. No puzzle addict could experience the miraculous intricacies and subtleties of the trail left by a program gone wrong. In the past, these interesting aspects of program testing have been dampened by the difficulty in rigorously extracting just the information wanted about the performance of a program. Now, however, sophisticated systems are available to relieve the programmer of much of this burden.

Testing for errors grows more difficult every year.
The previous sentence was an optimistic statement a half-century ago, but not because it was wrong. Over all these years, hundreds of tools have been built attempting to simplify the testing burden. Some of them have actually succeeded. At the same time, however, we’ve never satisfied our hunger for more sophisticated applications. So, though our testing tools have improved, our testing tasks have outpaced them. For examples and explanations, read about “preventing testing from growing more difficult.” 

If you're as interested in errors as I am, you can obtain a copy of Errors here:

ERRORS, bugs, boo-boos, blunders
SaveSave