mirror of
https://github.com/zaphar/jeremy.marzhillstudios.com.git
synced 2025-07-23 19:49:57 -04:00
66 lines
3.0 KiB
Markdown
66 lines
3.0 KiB
Markdown
|
Measuring Developer Skill
|
||
|
|
=========================
|
||
|
|
|
||
|
|
A recent article about a pycon keynote by Jacob Kaplan Moss brought an
|
||
|
|
interesting question to my attention. How exactly would one measure
|
||
|
|
developer skill. It's interesting because the question itself is hard
|
||
|
|
to define properly. For one thing there are a lot of dimensions to
|
||
|
|
measure skill in. And developers hold many different roles in a team.
|
||
|
|
DevOps, Architects, Maintainers, BugFixers, Automators. Each of these
|
||
|
|
requires different skillsets. My brother who is also a developer made
|
||
|
|
the comment that skill measurements are useless without an
|
||
|
|
expectation. Against what benchmark are we measuring the skill?
|
||
|
|
|
||
|
|
I'm not at all sure there is any answer to this question that even makes
|
||
|
|
sense. But perhaps we can make some useful progress if we narrow the scope
|
||
|
|
a little.
|
||
|
|
|
||
|
|
What if we just tackled the question of API design?
|
||
|
|
|
||
|
|
Measuring a single dimension of coding skill.
|
||
|
|
---------------------
|
||
|
|
|
||
|
|
API quality is one dimension on which you could measure developer
|
||
|
|
skill. For a measurement of API quality to be useful though it has to
|
||
|
|
be more than some elusive abstract idea of elegance or beauty
|
||
|
|
though. What would make a truly useful measurement of API quality?
|
||
|
|
|
||
|
|
When designing an API one of the commonly held goals is the principle
|
||
|
|
of least surprise. You want to build an API that makes sense to the
|
||
|
|
user and isn't bewildering in it's behavior. This seems like it
|
||
|
|
should somehow be measurable right? But how do you measure how
|
||
|
|
"surprising" an api is?
|
||
|
|
|
||
|
|
Another commonly held goal is the to reduce complexity. An API's
|
||
|
|
purpose is to hide a lot of the complexity in a problem and make it
|
||
|
|
comprehensible to the user.
|
||
|
|
|
||
|
|
Both of these ideas at there core are centered around the concept of ease of use.
|
||
|
|
What we need is a way of quantifying Ease of Use in a measurable form.
|
||
|
|
|
||
|
|
# WTF/dev/day #
|
||
|
|
|
||
|
|
Really the reason we care about ease of use is because it reduces
|
||
|
|
frustration when we consume an API. So the *number* we are trying to
|
||
|
|
reduce is the count of WTF's devs encounter when working with an
|
||
|
|
API. We could simply measure how many times a developer encounters a
|
||
|
|
WTF when using an API. But this is of course inherently biased. We
|
||
|
|
need a way to control or reduce that bias if we care about creating a
|
||
|
|
measurement that is useful to an industry.
|
||
|
|
|
||
|
|
What factors do we need to control for in a measurement of coding skill?
|
||
|
|
------------------
|
||
|
|
|
||
|
|
Assuming we are measuring in a single dimension we are going to have to decide
|
||
|
|
exactly what factors are important and what factors should be controlled for in
|
||
|
|
our measurements.
|
||
|
|
|
||
|
|
People with stockholm syndrome are obviously going to experience less
|
||
|
|
wtf's per day than someone new to an API. The quality of the API's
|
||
|
|
documentation also affects the wtf count. Which brings up a
|
||
|
|
question. Is the documenation a part of the API? should it be a
|
||
|
|
factor in our measurement or should it be restricted to code alone?
|
||
|
|
|
||
|
|
We will encounter hundreds of these factors for any dimension we
|
||
|
|
attempt to measure. Collecting useful data here is a hard problem.
|