The other day I was working on updating my visuals to describe the systems delivery lifecycle (SDLC) for software. Three new depictions arose.
-
Why does the SDLC need a better visual?
-I'm doing this because I've never been afraid of change, and the visuals still in use today depicting the phases of delivery are disturbingly incorrect to the modern lifecycle. Take for instance a few common illustrations:
-
Waterfall, cyclical, even the dreaded CI eternity symbol, all crap as far as I'm concerned. The linear fallacy is just too much to bear.
-So I got to work on the whiteboard, to play around with some ideas. I really tried to stick to the "design, build, test, deploy, manage, monitor" phase reductions to simplify my life before round 2 coffee.
-
Is the SDLC Just a Fractal?
-The idea that even a man-made structure like wire transmissions* could elicit fractal patterns just like nature got me thinking about other man-made structures...software and processes...are there fractals in what we accomplish, despite our incredibly synthetic rules and predisposition for straight lines? I've always loved this painting:
-
Fractals have changed my thinking about perceived complexity as a result of a simpler algorithm. There are fractals in all of us, our heartbeat graphed out show how nature may just be surface complexity to our perceived reality.
-So I drew out the 'waterfall' (yes, pun intended) like a Kanagawa painting:
-
* Reference to 'Hunting the Hidden Dimensions' documentary on Mandelbrot's early work at IBM related to SNR telephone clarification algorithms.
-If you haven't read 'An Eternal Golden Braid', at least watch a few moments of the following video, I implore you.
-[embed]https://www.youtube.com/watch?v=j5ty2s1TZvE&list=PL068ES-0ca9CSIp5OPGI5RXB3k5XgYRxF[/embed]
-So this works well for categorizing the levels of detail that a large-scale operation like rolling a new idea out into the market requires, but there's no link to the continuous aspects of the process. Next.
-How Is the SDLC Like Digital Public Transit?
-After going back to the 'wheel' mentality, I realized that there are just some parts of the SDLC that don't always connect to each other, or at least not every time you complete a full cycle. Things like 'build' in a TDD shop do not block 'testing', so forcing everyone to go through the 'build' phase is wrong.
-If instead we were to write out each 'phase' of the SDLC in clockwise fashion in a circle, then draw directed vertices, we'd get something like a subway map, like this:
- 
So that's interesting, certainly more palatable than the fractal illustration for most people, and shows a bit of symmetry, even if just as a shallow concept. The thing I like the best about this one is that you begin to see logical byproducts result from visualizing the paths through this state diagram.
-If you draw a line from right before 'design' until right after 'test', you'd see how the three design/build/test phases that traditionally developers would do and operations wouldn't touch are separate from the deploy/manage/monitor phases, which mirrors how proficiency in either (but not both) roles usually decides which set of phases you'll be spending most of your time on as a member of a 'DevOps' team, either primarily a developer or an operations person.
-Does the SDLC Need Some Taoist Whoop-Ass?
-Then things got really weird. I realized that the closer each of the phases are depicted to one another, the easier it was for me to keep in mind that it was possible (sometimes fortuitous) to jump from phase to phase as organically happens in real DevOps / agile teams.
-Instead of putting the elaborations (curls, paths) on the outside of the continuous structure (circle), they should be inside it. At first I drew it like a '69' which I later remembered has cultural connotations that are best avoided. Flipping it around, only later when I was out to coffee with some co-workers did someone point out that it looks like the Taoist Taijitu (paisley) symbol. That endeared it to me further:
- 
(From top-right in clock-wise order: design, build, test, deploy, manage, monitor)
-Any time I can work in some pseudo-science or eastern philosophy into a technical topic, the added mystical sense is a bonus as far as I'm concerned. You come off looking all cool and a bit nuts at the same time.
-Further Detail on Why I Don't Like Existing SDLC Paradigms
-Formally, my problems with established paradigms for how to visualize the SDLC are:
--
-
- The left-to-right approach as a one-direction trip is inaccurate to how effective people work together, not separate; there are loopbacks thanks to humans -
- The 'fix' to take a waterfall process and just wrap it around on itself to make a continuous process just seems lazy and incomplete -
- Most depictions have no symmetry and no emergence of 'DevOps' role separation -
People will use what makes sense to them. I will continue to elaborate these ideas until they bore me.
diff --git a/_posts/2015-11-06-iteration-maths-and-continuous-delivery.html b/_posts/2015-11-06-iteration-maths-and-continuous-delivery.html deleted file mode 100644 index b7a8443..0000000 --- a/_posts/2015-11-06-iteration-maths-and-continuous-delivery.html +++ /dev/null @@ -1,66 +0,0 @@ ---- -layout: post -title: Iteration, Maths, and Continuous Delivery -date: 2015-11-06 18:09:50.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: [] -tags: [] -meta: - _edit_last: '1' - _oembed_55f1bd94a645c3046c953fc1a0e0052c: - _oembed_time_55f1bd94a645c3046c953fc1a0e0052c: '1446814765' - _oembed_931d9b2a015456ee359183b0e8933ee2: - _oembed_time_931d9b2a015456ee359183b0e8933ee2: '1450446910' - dsq_thread_id: '4612245753' - _oembed_144a4922023a7ab1b65d5d843de61619: - _oembed_time_144a4922023a7ab1b65d5d843de61619: '1484611494' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2015/11/iteration-maths-and-continuous-delivery/" ---- -Recently I’ve been reading "Godel, Escher, Bach: An Eternal Golden Braid". I also love to stare at trees, and this morning while waiting for the train, something occurred to me: “What is a tree without time?”
-I’ll get to computer stuff in a bit, but follow me for a minute here.
-Things take time to develop
-We live in a multi-dimensional universe, and one of those dimensions we perceive as the passing of time. Strangely for its inhabitants, time appears to only go in one direction. Concurrently, iteration seems to have a “direction” as well. Just as with a single-dimensional line, you can iterate forwards and backwards. Time’s arrow shows us that’s how it works for us. The concept of “direction” in iterates is also well understood, but there seems to me to be an obvious correlation between time’s direction and iteration.
-Back to the tree. My point is that the branches are just iterations of a simple instruction set. They look different, but it’s just a mixture of randomness and iteration. Like this picture of a hair.
-
Watch the video below, but please come back and read the rest of my ramblings after you see this guy reproduce what looks like hair follicles in less than 10 lines of code:
-[embed]https://youtu.be/PBDQL7hp7gk?t=1h1m[/embed]
-Give me something really WILD to ponder
-Warning => Conjecture: I am not a scholar, just a really interested party.
-Also, this sentence is not true.
-Nowhere in the universe we perceive exists an example of the opposite of iteration (let’s call that idea “statics”). We know that all physical things are part of spacetime. Quantum “jitters” show that even bodies at rest are not truly at rest. Empty space is never truly empty, virtual particles pop in and out of existence constantly. Could this just be way of spacetime alleviating itself of the need to iterate the physical dimensions to keep balance with the arrows in what we perceive to be non-physical dimensions? Or possibly how fractional geometry approaches the Euclidian sticky points in our perceptive capabilities, by iterating continuously ad infinitum? Boom.
-How does a book about hair styles help me with a DevOps strategy?
-“An Eternal Golden Braid” makes you think, hard. In most cases, that’s a good enough reason for me to read something. And I hate reading, but I love learning. That’s my opinion, go make your own up.
-A key concept to keep in mind if you read this book is how entangled things get once you start asking very granular questions about relationships between the observable universe and mathematics (maths), maths and a definition of the self, and how far we can reduce a concept before it satisfactorily models the phenomena we uncover with math, science, and philosophy.
-Iteration is present in maths, starting with the concept of zero, something it took us a long time as humans to identify. Take none, add one. Keep doing that until you reach infinity. Presto! Thanks to the simple concept of iteration, when mixed with literally nothing (zero), exposes the concept of infinity.
-"DevOps" is an amalgamate term now, involved in everything from rapid iteration to deployment methodologies, distributed systems architecture to “soft” skills amongst team members. It’s all tangled up, which is not a bad thing necessarily. We need cross-disciplinary thought to help expand on how the practices we implement fit together, complement each other, and even conclude something which only causes problems and needs to be excused permanently (think M$ DCOM, Clippy, and maybe even gun control). I don't like any of them.
-How does this relate to continuous delivery?
-I am a man who prefers science whenever possible, but I’ll settle for less than scientific if it has promise and until it is disproven. A few taking point came to me this week:
--
-
- Fractional iterates lead to the concept of continuous parameters. Continuous parameters, isn’t that the software we’re building? Or is it us, the people AND the product? This sounds like some of the thinking we have around continuous delivery, loops of deployment and redesign. -
- Any time infinity walks in the room, everything goes to hell and no one likes a know-it-all at a party. Paradoxes abound. The concept of continuous delivery seems a bit like Dante’s seventh circle to me, in that nothing ever ends, it just goes on and on. -
- Like spacetime, the patterns we use and the goals we want to reach are not separable. Better practices lead to better results. -
- Like every tree, every good idea takes time, iteration, and a bit of [pseudo]randomness to develop into something great. -
Use whatever processes, tools, and timeframes you need to, but exploring the philosophical underpinnings of continuous systems is kind of necessary if you want to do things better over time.
-Can you give me something more practical so I can go home now?
-Okay, sure. Here are a few things I’d do if I was a development or a product owner with a team of pangalactically skilled people on my team:
--
-
- Have everyone read this book, or at least this MIT lecture video series on the book -
- Allow members of your team one day a month to play with any idea they want to, the sole purpose being that it is a test of how they manage themselves and accomplish personal goals -
- Iterate over the design of your processes, not just the design of your software; anything that’s working well, f*ck with it. Anything that’s not working well, replace it with something better. -
- Assign one person per week to be an agent of change by asking them to cross-train with other members of your team on one or two things they don’t know about. As the Joker famously said, “Introduce a little anarchy”, which I have seen work very well to upset the status quo in practice. -
Good luck. Let me know how it goes.
diff --git a/_posts/2015-11-09-swagger-oadf-an-insiders-view.html b/_posts/2015-11-09-swagger-oadf-an-insiders-view.html deleted file mode 100644 index 633911d..0000000 --- a/_posts/2015-11-09-swagger-oadf-an-insiders-view.html +++ /dev/null @@ -1,43 +0,0 @@ ---- -layout: post -title: 'Swagger => OADF: Insider View' -date: 2015-11-09 07:06:34.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- API -- API design -- API documentation -tags: -- Apache -- OADF -- OAI -- Swagger -meta: - _edit_last: '1' - _thumbnail_id: '26' - dsq_thread_id: '4387268246' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2015/11/swagger-oadf-an-insiders-view/" ---- -I still don’t get it. After multiple clarifications with a friend about the recent donation of the what-once-was-only-Swagger API description format to the OAI (Open API Initiative), I'm still confused as to why the contribution of the impartial format without what would be very one-sided contributions of tooling around the format is not the safest step to getting a long-term standard for everyone to benefit from into the public domain. I argue, it is, even if we don’t know how it will turn out.
-Not at the cool kids table, what do I know?
-I get that some people are feeling confusion, frustration, shock, and even disgust. I also hear people expressing relief, clarity, gratefulness, and vision. The world has far fewer thought-leaders and way more people who want standards they can build cool shit with, make some money and some good happen, and ultimately go home knowing things will generally work as designed again tomorrow.
-I’m about to state some things, job whatever. I had no say in the way any of this played out, but we need someone to clarify that the insides of my corporate office is filled with good, fair, and driven people. This is not an evil empire, unlike some other soft competitors, this is where I work. Respectfulness and honesty are two of the Apache Way tenants, two that ring true for me and those I work with.
-What are the facts IMO?
-Fact: Certain entities owned the rights to the Swagger technology and brand, which for better or worse started as a format plus a cool name mingled together. It eventually became so useful that the format was open-sourced. Later, a company was able to navigate the financial and political mess that has been the API description format landscape for years, producing an API ecosystem standard as significant to real implementers as other open standards like HTTP, JPEG, and . Still, the brand has monetary value but the spec has much broader universal value, and so the spec was donated.
-Fact: No one could “sell” the Swagger format is because it was already open source, and though the brand was still legal property, it has been managed with no misinformation or clandestine story behind it. When technology becomes part of a commons, it no longer makes sense to have one brand dominate the tooling and the vendor ecosystem. A separation of the format and the brand was necessary. To convolute the new OADF (Open API Description Format) by including tools that specific vendors donated but not including others would have been even worse an idea than not separating out the format from the brand, and persisting to do so doesn’t prevent these tools from being just as useful and successful as they were before the OAI changes.
-Fact: People that truly helped the Swagger ecosystem get to where it is today deserve due credit for what they have done. Their contributions continue to be pivotal to the OADF because without a community, big companies that are already strong-arming the media with boku marketing budgets and completely horse shit press releases will take the OAI over and rule the connected world, one bad choice at a time. The community must maintain leadership in the OAI, and if there's no room at that table for proven thought-leaders who have already spent years improving and evangelizing the importance of a format like Swagger, then shame on those who say they are leaders but are to busy making ends meet to ask for the right kind of help or spearhead truly pivotal contributions to the open web.
-Fact: SPDY was a great start to fixing something, and I liken the work done at Google with the help of the whole industry to elevate much of the great ideas there into the formal HTTP 2.0 spec. Same with WebRTC, if only we could get proprietary chipset manufacturers to be adults and ship more than one optimized instruction set for video decoding on their silicon communion wafers. Much like the SPDY-to-HTTP2 evolution, the need for a standard API description format is long overdue, the OADF being a necessary leap forward which should have happened years ago.
-What are we left with in the OADF?
-I admire people who stand up for what they feel is right and who are vocal about what happens when things don't go the way they expected them to. A brand is monetarily valuable, search and replace on the internet is not possible (I get that), and decision-makers usually make the decisions they are hired to make. That’s just how that works.
-In the meantime, we have a standard open API description format now, frankly better than all the rest of them, and we can build on it just like all the other foundational technology that makes up the connected world we live in today. Brands are bullshit anyway. It's the meaning of the thing behind the name that matters.
-Peace, love, and internet for everyone.
diff --git a/_posts/2015-11-10-advocation-for-open-source.html b/_posts/2015-11-10-advocation-for-open-source.html deleted file mode 100644 index a79b319..0000000 --- a/_posts/2015-11-10-advocation-for-open-source.html +++ /dev/null @@ -1,41 +0,0 @@ ---- -layout: post -title: Advocation for Open Source -date: 2015-11-10 05:18:44.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- open source -tags: [] -meta: - _edit_last: '1' - _wp_old_slug: an-advocation-for-open-source - dsq_thread_id: '4519347103' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2015/11/advocation-for-open-source/" ---- -Getting people to understand the true value of open source goes beyond just making easily digestible bullet points. I don't mean the hyper-commercialized version of open source that dominates the software market today. I mean that open source software is a way in to the mind and heart of a programmer.
-Programmer != "developer"
-Programmers love code. They love to fix things. They love when something can be expressed in concrete terms. They love other people, though not all of them know that they express this through their code. They hate when something doesn't work properly, just like a true engineer does. They dislike bullshit, in any form. They hate when something that should be free, isn't.
-
The term "Developers" on the other hand is a concept that certain people invented, constructed, to create a marketplace of people that do first instead of consider first. Developers, not code, have been the new tech market for many years. Even well-meaning technical people sadly fall prey to the wholesale commoditization of humans who speak in computer languages and are easily caught up in the dark ocean of enterprise software.
-I wish all the developers in the world well as I sail away and find a more honest place to keep my self and my code.
-This guy worked as a developer, then he quit work
-I have contributed to many open source projects over the years. Some I have taken too personally, some I have ignored, and some I squandered. Sadly, for me and this post, it cannot be verified, as I have used many faux-names and anonymous accounts to do so. I can't even remember or trace them even if I tried. [heartache] [pause] [sigh]
-In my youth, I didn't realize how important contribution was. I thought that the concept of the 'self' was something to be avoided, ignored. Since that time, I understand how wasteful that viewpoint was. Contribution is a requisite now.
-Open source software is the language of people who want to work together.
-The problem with money
-There's something inherently wrong with a company that doesn't regularly contribute back to the community, something very wrong and very rotten. Money is a really effective motivator. It is often also used as the only indicator of value in a marketplace. Shame on us for not looking deeper, because there are many metrics to human behavior and interest, the last of which is where people put their money. Ask anyone in sales, they'll tell you that people's interest is ultimately, but not initially gauged by their financial commitment. People will tell you if your thing, whatever that is, is bad or worth investing in; you just have to listen to the signs. And there's no better way to do that than open source contributions as a metric for the veracity of your thing in the market..
-I advocate for open source
-One theme that emerges from my years into software is that contribution is key. Commercial, open source, whatever. Teams are made up of people, and to do something great, the more engaged and honest people you find, the better off you'll be.
-Side note: It is not enough to treat a non-profit as your contribution. The maintstream NPO mentality is that money is only necessary when it's needed, desperately, in salaries, in budgets, and in planning. People are not created equal. We start at different places, with different perspectives, and claw our way to an egalitarian place. Some people have to work harder than others.
-It is the job of those who don't work as hard to help out those who work too much.
-That is the spirit of open source to me. To help. That's why I advocate open source now. It is help that causes people to put there money where their mouth is, in your product. To make money is to help first; the software equivalent of that is to contribute to open source, whatever way you can.
diff --git a/_posts/2015-11-11-defrag-2015-beforemath.html b/_posts/2015-11-11-defrag-2015-beforemath.html deleted file mode 100644 index 1ce4ee9..0000000 --- a/_posts/2015-11-11-defrag-2015-beforemath.html +++ /dev/null @@ -1,28 +0,0 @@ ---- -layout: post -title: Defrag 2015 Beforemath -date: 2015-11-11 05:10:05.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: [] -tags: [] -meta: - _edit_last: '1' - _wp_old_slug: defrag-2015-pre-chat -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2015/11/defrag-2015-beforemath/" ---- -It's interesting to see themes change year after year. At an event like Defrag or Gluecon, it's hard to ignore the voraciousness of curiosity. Tinkering.
-Of course I'm interested in pretty much all of the keynotes, especially Sam Ramji. Pivotal and all it's incarnations has been of interest to me for for years, the foundation just caught my attention because of its focus on contribution.
-Fact is, the back-half of the first day is all about APIs. That's where I'll be. That and the SmartBear booth between sessions. There are big differences between last year and this year, and not just my hair and shoes.
-
I come out here to have meaningful, real conversations. About everything, not just APIs, not just technology. I hope to elicit stories from people about their own experiences and challenges with the evolving connected landscape. As much as IoT isn't a revenue thing right now, it is still in the forefront of my mind, on my radar and on the tip of my tounge.
-Come find me.
diff --git a/_posts/2015-11-14-defrag-2015-legit.html b/_posts/2015-11-14-defrag-2015-legit.html deleted file mode 100644 index 5ec1429..0000000 --- a/_posts/2015-11-14-defrag-2015-legit.html +++ /dev/null @@ -1,48 +0,0 @@ ---- -layout: post -title: Defrag 2015 == Legit -date: 2015-11-14 13:19:02.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- API -- event coverage -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '51' - dsq_thread_id: '4410110242' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2015/11/defrag-2015-legit/" ---- -
Defrag is legit. By "legit", I go with the urban dictionary definitions in that it is "real", "authentic", "truthful", generally a good thing.
Who am I? Just a guy who goes places and interacts with other real people, like this guy, Andy Rusterholtz who isn't even on Twitter yet.
-Keynotes (a.k.a. new friends)
-Ramez Naam illustrated how our conscious perception of the world around us is very much a function of both sensory input and our memory of past input mixed together, never a perfect raw clear representation reality. He followed that up with proof that these squishy memories are entirely transmittable onto silicon. Want someone else's memories? They'll come mixed with yours, but we can do that now. He came in full Philip K. Dick style. This guy is within calling distance of Orson Scott Card via Wasteland 2 and Brin. Legit.
-
Mary Scotton put the whole keynote floor to rest with the depths of her compassion for considering the inequities of the industry around sexism, racism, and greed. Being inclusive is a responsibility we all share in common as humans who work for companies. Legit. She answered each of my quotes with a witty twitter mention to the original source of the quote or idea. Inclusive, ask Ben or anyone else she talked to, ever. Legit. "I don't have to have the same kind of talk with my son that African American parents have to have with theirs...", as she has a still photo of the Rodney King tragedy. Legit.
Bilal Zuberi. Great research. Great oration. Now that I've looked him up, typed hist name out, and referenced it in many conversations since yesterday, I'll remember it well. His ideas for how much we should invest in reaching higher as a human race through technology and how the best leaders are formed; no problem remembering them now either. Are we really trying to get liquor to your door faster via mobile app, or should we maybe cure cancer first, then celebrate after? Legit.
-
Lorinda Brandon. Mindful tech. It is socially irresponsible to let courts rule in favor of letting upskirt videos be taken, protected under "free speech" because they are recorded by a phone. What does that make a phone? Things don't go away once you share them. Also, she put the kibosh on Google images as a way to help law enforcement crack down on illegal pools but not on illegal acts of law enforcement. Legit. She makes advocating for privacy the new gold standard for how to show which institutions and governments are leaders and which ones aren't. Legit.
Lisa Kamm. Legit. Right after Lindy challenged Google's propensity for privacy snarls, Lisa bounces back by showing us all that we need to get the fuck off our phones while in transit. We are not efficient at either when we do both at the same time. The only demonstration in her talk was her demonstrating how to navigate complex topics gracefully. Legit.
-Kin Lane. A great mind behind honest ideas like APIware and APIs.json, a new format for how to describe the API lifecycle, something he invents in his spare time. Legit. Thinker of thoughts. Most terse person in the world if he wants to be, recently so about the OAI and about Swagger. You just gave all the students at Defrag (myself included) a map for how to build businesses around API tech. Legit.
-[Transparency. I was not able to make a few of the sessions that I only heard people talking about afterwards. Duncan's talk, for instance.]
-Sam Ramji, we hope your shoulder feels better. Sad that you couldn't come inspire us. Hello world demos aside, I will look for some time with you like I did with Phil Windley where we can talk about some stuff that is important.
-Anya Stettler. Renegade developer evangelist at Avalara. What the hell is a tax company doing paying for a badass, beautiful brain such as hers to come and speak? Same thing that Capital One is doing by convincing Lorinda Brandon to join their team. It's called financial technology, and it took over everything recently, did you know? Anya is shorter than I am, had more to drink than I did, and still kept going back and forth with Mick longer than I could. She knows who her people are. Legit.
-David Nielsen. I missed his talk entirely, sat across from him a lot in two days, took no notice as he slept while sitting upright at "Indian" dinner with me and Emmanuel Paraskakis, only to wake up and lay down some serious story about working in India himself which to me at least made the already not-so-Indian food seem a less authentic. I still ate it up, his company and the food. Legit.
-I could go on, but not really because I was busy having other conversations and missed all of day one and some of day two. Sad for me. Not legit. Also not legit, I missed the cloud foundry meetup last night.
-The Legit Thing to Do: Say Thank You
-Eric Norlin. Incubator. There were students all over this conference. A true sign of legit. I'll post on this topic more tomorrow, better timing for them and for Maria at StartupDigestCO #GSB2015. Eric and these kids are enabling the next great thinkers and business owners to connect up and are helping them make good choices about their careers from day one. Legit.
-Kim Norlin. Consummate professional host and organizer. Just being in her presence make me know I will never hold a conference myself. I'll just ask her to either do it for a very hefty sum or a referral for someone else who can. She closed one of the bars down when there was no point in having it open anymore. Legit.
-What's Next?
-I let some students speak their peace about getting poached like river trout by VCs and sponsors with dollars. I'll post that tomorrow when I've had a few hours sleep between today and tomorrow, wait, no, today.
-Also, I'll be speaking at APIstrat in Austin next week, asking the question "how early is too early for childhood development around digital devices and #techlife?"
-Peace to us all, especially those involved in the tragedy tonight which writing this article has acted to help me avoid breaking down about.
--
diff --git a/_posts/2015-11-15-hackcu-an-example-of-student-leadership.html b/_posts/2015-11-15-hackcu-an-example-of-student-leadership.html deleted file mode 100644 index 6b13daa..0000000 --- a/_posts/2015-11-15-hackcu-an-example-of-student-leadership.html +++ /dev/null @@ -1,44 +0,0 @@ ---- -layout: post -title: 'HackCU : An Example of Student Leadership' -date: 2015-11-15 07:15:12.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- hackathon -- open source -tags: [] -meta: - _oembed_5c5ed7960fa1eccbb12778200968503e: - _oembed_7a57114967e6a5abe93d3905c61eff15: - _oembed_time_7a57114967e6a5abe93d3905c61eff15: '1447550696' - _edit_last: '1' - _oembed_time_5c5ed7960fa1eccbb12778200968503e: '1450140909' - _thumbnail_id: '156' - spacious_page_layout: default_layout - dsq_thread_id: '4522239444' - _oembed_f3a7d02826e15987eb8c0e3c8a05ba6e: - _oembed_time_f3a7d02826e15987eb8c0e3c8a05ba6e: '1484639619' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2015/11/hackcu-an-example-of-student-leadership/" ---- -
In an unused downstairs side room of a hotel, I listened to students from the University of Colorado express their desire to change the world, and their concerns about commercial interests moving in to poach talent from hackathons.
-Alex Campbell, Alex Walling, and Nika Shafranov.
-[embed]https://youtu.be/vo-lt7PK5ww[/embed]
-[Download as MP3]
-HackCU is a Boulder-based hackathon incubator program that seeks to educate, inspire, and connect local students to technology and hands-on skills. It is at the cutting edge of technical education. It is student-led. It is well intentioned.
-It is a way for students to arm themselves in the digital workforce when they can't trust the technical market to treat them properly as employees. They teach each other how to code, design, and prove the value of their own technology.
-I think this is great. These are the right people to be running this sort of thing. I'll be keeping my eyes open about hackathon politics in the local area thanks to my conversations with these fine engineers.
-My goal in engaging the two Alex's and Nika, with help from Maria Sallis of StartupDigestCO and Lorinda Brandon's wise words, was to assist these students in any way a storyteller like me can: network the right people together, understand their challenges, and help them tell that story to a wider audience.
-Much like the open source community (which I am far less a part of than I'd like to be), the communities of student-led hackathons are a brilliant place to hang out and listen, intellectually stimulating and ethically challenging. My own exploration of this space will take time.
diff --git a/_posts/2015-11-18-quality-means-not-accepting-crap.html b/_posts/2015-11-18-quality-means-not-accepting-crap.html deleted file mode 100644 index c4a5a67..0000000 --- a/_posts/2015-11-18-quality-means-not-accepting-crap.html +++ /dev/null @@ -1,42 +0,0 @@ ---- -layout: post -title: Quality Means Not Accepting Crap -date: 2015-11-18 07:06:38.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- quality -- software -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '74' - dsq_thread_id: '4452948388' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2015/11/quality-means-not-accepting-crap/" ---- -Software. Hardware. Things. Opinions. Places. Excuses. Ideas.
-Anyone can produce a cheap "affordable" solution. But details matter. How many cheap plastic things have broken in your hands unexpectedly, and were entirely disappointing in that moment?
-My AirBnB is not that. I knew "quality" when I saw it. You can tell someone lived in this thing and made it convenient for them, then handed it off to you. That's quality, making something that meets your own standards, then giving it to someone else.
-
I travel a lot, enough to know what matters on a trip. Leg room on the plane. Working wifi. Power plugs, everywhere. Politeness. Clean bathrooms. Details matter.
-Conversely, a $300/night hotel room only to have plugs too far away from the bed, lamp toggle buttons that take so much effort to push that you push the lamp over, light switches that are harder to find than Carmen Sandiego; the annoyances all add up too. The lights in this camper that I'm staying in are easy to use and don't cause me to cuss.
-
Same with software, details matter.
-Quality software comes from people using their own product, living in it, fixing its flaws, and asking others how their experience with it is. In the tech industry, we call it "dogfooding" your own product. Believe me, it works.
-People intrinsically know "quality" when they experience it. They pick up a phone, it's heavy and solid, they think "that's quality". Conversely, they close a car door and it rattles or sounds hollow, they think "that's cheap". Even the sounds shipped with your mobile phone help to engineer your perception of the quality of the device.
-Quality is in the details.
-Oh, and BTW, I'd also rather put a constraint on myself not to over-drink and stumble into a $450/night on-premise room way too late at night to wake up on time the next morning. I have business to attend to. Knowing when to quit starts with looking at a ridiculous estimate and just saying no:
-
So, even at only 3 nights, this would have cost $1,350 just for a room I would be spending around 4-6 hours a night in, and not getting all the charms of an outside shower and condensation on the windows each morning. The AirBnB alternative for all three nights, just 60% as compared to JUST ONE NIGHT AT THE SHERATON!
-
I should have remembered to check the crime overlay though, but Uber is a cheap solution to that problem:
-
I’m taking on the task of building an automation system for some of my online social engagement. Since I am not such a Very Important Person (yet :), the absolute worst that can happen is that one of my friend/followers shares something racist or sexist or *-ist that I wouldn’t otherwise agree with. Bad, but I can at least un-share or reply with an "I'm sorry folks, my robot and I need to talk" statement. But this leads to an interesting question:
-What does it mean to imbue responsibility over my online persona to a digital system?
-It’s not really that bizarre of a question to ask. We already grant immense amounts of control over our online profiles to the social primaries (i.e. Facebook, Twitter, Google+). For most people, any trending app that wants access to “post to your timeline” is enough of a reason to grant full access to activities on behalf of your profile, though it shouldn’t. Every time you want to play Candy Crush or Farmville, you are telling King and Zynga that it’s okay for them to say whatever they want as if they were you to people in your network.
-The more of a public figure you are, the more your risk goes up. Consider that Zynga is not at all incentivized to post bad or politically incorrect content to your network on your behalf. That’s not the problem. The problem is when (not if) the company behind a game gets hacked, as did Zynga in 2011. It happens all the time. It’s probably happened to you, and you stand to lose more than just face.
-So what is the first thing to get right about automating social media?
-Trust and security are the first priorities, even before defining how the system works. Automation rules are great except for when the activities they’re automating do not follow the rules of trust and responsibility that a human would catch in a heartbeat. There is no point to automation if it’s not working properly. And there’s no point in automation of social media if it’s not trustworthy.
-For me at least in the initial phases of planning out what this system would look like, trust (not just “security”) will be a theme in all areas of design. It will be a question I ask early and often with every algorithm I design and every line of code I write. Speaking of algorithms, an early example of these rules go something like this (pseudo-code):
-F(auto-follow): - For(accounts = (new followers | in [list] | [search results] | [known queries]) - Given [accounts] with [golden keywords] in their description - AND who have a minimum of [2x more followers than me] - AND who have [golden keywords] in at least [3] posts no older than [2 weeks] - THEN auto-follow - -F(auto-share): - For(accounts = (in [list] | [known queries]) - Given [accounts] - When recent [share] contain [golden keywords] - AND [share] has been [highlighted] by [followers in common] - AND [share] does not contain [excluded keywords] - Add to [buffer] for [reshare] - AND send notification of future reshare to email - UNLESS already in buffer - OR daily personal reshare quota is reached - OR daily contributor-specific quota is reached-
-
diff --git a/_posts/2015-11-26-dont-insult-technical-professionals.html b/_posts/2015-11-26-dont-insult-technical-professionals.html deleted file mode 100644 index f2d3bb5..0000000 --- a/_posts/2015-11-26-dont-insult-technical-professionals.html +++ /dev/null @@ -1,40 +0,0 @@ ---- -layout: post -title: Don't Insult Technical Professionals -date: 2015-11-26 18:42:53.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- API -- quality -- testing -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '100' - dsq_thread_id: '4450923865' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2015/11/dont-insult-technical-professionals/" ---- -
Some vendors look at analyst reports on API testing and all they see is dollar signs. Yes, API testing and virtualization has blown up over the past 5 years, and that's why some companies who were first to the game have the lead. Lead position comes from sweat and tears, that's how leaders catch the analysts attention in the first place; those who created the API testing industry, gained the community and analyst attention, and have the most comprehensive products that win. Every time.
-There are snakes in the grass no matter what field you're in
-I recently had opportunity to informally socialize with a number of "competitors", and as people are great people to eat tacos and burn airport wait time with. Unfortunately, their scrappy position in the market pushes them to do things that you can only expect from lawyers and pawn sharks. They say they're about one thing in person, but their press releases and website copy betray their willingness to lie, cheat, and deceive actual people trying to get real things done.
-In other words, some vendors proselytize about "API testing" without solid product to back up their claims.
-I don't like lying, and neither do you
-One of my current job responsibilities is to make sure that the story my employer tells around its products accurately portray the capabilities of those products, because if they don't, real people (i.e. developers, testers, engineers, "implementers") will find out quickly and not only not become customers, but in the worst cases tell others that the story is not true. Real people doing real things is my litmus test, not analysts, not some theoretical BS meter.
-Speaking of BS meter, a somewhat recent report lumped API "testing" with "virtualization" to produce a pie chart that disproportionately compares vendors market share, both by combining these two semi-related topics and by measuring share by revenue reported by the vendors. When analysts ask for things like revenue in a particular field, they generally don't just leave the answer solely up to the vendor; they do some basic research on their own to prove that the revenue reported is an accurate reflection of the product(s) directly relating to the nature of the report. After pondering this report for months, I'm not entirely sure that the combination of the "testing" and "virtualization" markets is anything but a blatant buy-off by one or two of the vendors involved to fake dominance in both areas where there is none. Money, meet influence.
-I can't prove it, but I can easily prove when you've left a rotting fish in the back seat of my car simply by smelling it.
-What this means for API testing
-It means watch out for BS. Watch really closely. The way that some companies use "API testing" (especially in Google Ads) is unfounded in their actual product capabilities. What they mean by "testing" is not what you know as what's necessary to ship great software. Every time I see those kinds of vendors say "we do API testing", which is a insult to actual API testing, I seriously worry that they're selling developers the illusion of having sufficient testing over their APIs when in reality it's not even close.
-Why your API matters to me
-On the off-chance that I actually use it, I want your API to have been tested more than what a developer using a half-ass "testing" tool from a fledgling vendor can cover. I want you to write solid code, prove that it's solid, and present me with a solid solution to my problem. I also want you to have fun doing that.
-The API vendor ecosystem is not what it seems from the outside. If you have questions, I have honesty. You can't say that about too many other players. Let's talk if you need an accurate read on an analyst report or vendor statement.
-diff --git a/_posts/2015-11-27-when-you-are-really-invested-you-worry.html b/_posts/2015-11-27-when-you-are-really-invested-you-worry.html deleted file mode 100644 index 8a915b0..0000000 --- a/_posts/2015-11-27-when-you-are-really-invested-you-worry.html +++ /dev/null @@ -1,30 +0,0 @@ ---- -layout: post -title: When you are really invested, you worry -date: 2015-11-27 04:47:02.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: [] -tags: [] -meta: - dsq_thread_id: '4450565674' - _edit_last: '1' - _thumbnail_id: '104' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2015/11/when-you-are-really-invested-you-worry/" ---- -
I watched a couple of guys help each other today. One of them wanted to test out a dory by paddling it around the harbor before selling it. The other didn't want anything to do with the water on such a nice day.
-I only began to take notice when the other pulled up to the local dock hurriedly, backing in to a double-spot closest to the water. Since we share a share a small neighborhood with him, I said hi and asked how his Thanksgiving was going. He told me fine and what his other was doing and how long it would take him to appear in the cove.
-In previous years, I might have lauded how awesome it was that his elderly friend was rowing around in open water, but today I quickly and carefully responded by saying "oh" and asking him "how do you feel about that"? He sort of muttered quietly first and then said loudly "oh, it's fine, he does things like this all the time", and wouldn't look me in the eye. After some chit-chat we moved off each other's company, and in about 10 minutes, his friend's two-person row boat peaked out from behind one of the lobster boats docked in the bay, with a guy in it, rowing slowly and steadily.
-After navigating past the dock and to the rocks where the truck was parked, they carefully collected the small dory out of the water and navigated it up into the truck bed. The rower then proceeded to fasten the protruding boat to the truck with ropes and carabiners while the driver stood patiently out of the way enjoying the 50 degree holiday weather, free of anxiety and grateful for the salty harbor air.
-When two people live together for the better part of their lives, like an odd couple mirroring the dynamics of other married couples, they develop a deep emotional connection to each other, the sweet bitterness of co-dependency. It is the bond formed between people who see more and more into each other, the wonders and the flaws, the longer they captivate each other's curiosity and souls.
-The best part of it is, there are dozens of examples of this fine form of relationship in my neighborhood. Men devoted to each other, women who have known from their first meeting that they were meant to live life together, and every other kind of relationship you can think of too. Single moms. Single dads. Parents of young and old. Grandparents. Young parents. Parents twice and thrice over. Kids made of the most creative and kind things in the known cosmos. Couples, singles, veterans, retirees. Humanitarians. Artists. Engineers.
-It takes all kinds. It takes these guys, to make a world. We're better for having them in it. They help us remember to follow the path of love back to each other.
diff --git a/_posts/2015-11-28-talk-api-strategy-the-next-generation.html b/_posts/2015-11-28-talk-api-strategy-the-next-generation.html deleted file mode 100644 index 616fa85..0000000 --- a/_posts/2015-11-28-talk-api-strategy-the-next-generation.html +++ /dev/null @@ -1,53 +0,0 @@ ---- -layout: post -title: "[Talk] API Strategy: The Next Generation" -date: 2015-11-28 04:10:20.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- API design -- automation -- hackathon -- open source -- SDLC -- software -- testing -tags: [] -meta: - _edit_last: '1' - _oembed_c0d4743f48fd055b41385a129fcc04b6: - _oembed_time_c0d4743f48fd055b41385a129fcc04b6: '1453415962' - _thumbnail_id: '110' - _oembed_86f52fe5e2af2318cf26bc75b52a31be: - _oembed_time_86f52fe5e2af2318cf26bc75b52a31be: '1450143897' - dsq_thread_id: '4452943313' - _oembed_89665e772b4bd1c1276347f4ecf52198: - _oembed_time_89665e772b4bd1c1276347f4ecf52198: '1484590379' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2015/11/talk-api-strategy-the-next-generation/" ---- -I took the mic at APIStrat Austin 2015 last week.
-A few weeks back, Kin Lane (sup) emailed and asked if I could fill in a spot, talk about something that was not all corporate slides. After being declined two weeks before that and practically interrogating Mark Boyd when he graciously called me to tell me that my talk wasn't accepted, I was like "haal no!" (in my head) as I wrote back "haal yes" because duh.
-[embed]https://www.youtube.com/watch?v=3cFDWAYM-78[/embed]
-I don't really know if it was apparent during, but I didn't practice. Last year at APIStrat Chicago, I practiced my 15 minute talk for about three weeks before. At APIdays Mediterranea in May I used a fallback notebook and someone tweeted that using notes is bullshit. Touché, though some of us keep our instincts in check with self-deprecation and self-doubt. Point taken: don't open your mouth unless you know something deep enough where you absolutely must share it.
-I don't use notes anymore. I live what I talk about. I talk about what I live. APIs.
-I live with two crazy people and a superhuman. It's kind of weird. My children are young and creative, my wife and I do whatever we can to feed them. So when some asshole single developer tries to tell me that they know more about how to build something amazing with their bare hands, I'm like "psh, please, do have kids?" (again, in my head).
-Children are literally the only way our race carries on. You want to tell me how to carry on about APIs, let me see how much brain-power for API design nuance you have left after a toddler carries on in your left ear for over an hour.
-My life is basically APIs + Kids + Philanthropy + Sleep.
-That's where my talk at APIstrat came from. Me. For those who don't follow, imagine that you've committed to a long-term project for how to make everyone's life a little easier by contributing good people to the world, people with hearts and minds at least slightly better than your own. Hi.
-It was a testing and monitoring track, so for people coming to see bullet lists of the latest ways to ignore important characteristics and system behaviors that only come from working closely with a distributed system, it may have been disappointing. But based on the number of conversation afterwards, I don't think that's what happened for most of the audience. My message was:
-Metrics <= implementation <= design <= team <= people
-If you don't get people right, you're doomed to deal with overly complicated metrics from dysfunctional systems born of hasty design by scattered teams of ineffective people.
-My one piece of advice: consider that each person you work with when designing things was also once a child, and like you, has developed their own form of learning. Learn from them, and they will learn from you.
-diff --git a/_posts/2015-11-30-unanticipated-editorial-control.html b/_posts/2015-11-30-unanticipated-editorial-control.html deleted file mode 100644 index 29f8725..0000000 --- a/_posts/2015-11-30-unanticipated-editorial-control.html +++ /dev/null @@ -1,32 +0,0 @@ ---- -layout: post -title: Unanticipated Editorial Control -date: 2015-11-30 18:05:46.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: [] -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '115' - dsq_thread_id: '4452944057' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2015/11/unanticipated-editorial-control/" ---- -
I took down someone's blog post of an event I was at two weeks ago.
-All I did was to praise one of the panelists for the amount of mic-drop-esque quotations attributed to her, clear misquotes to anyone who knew how she speaks and who was paying attention. Not too many in both categories, sadly.
-This is what social media and marketing gets away with all the time though. Content that is rarely verified by others in the know.
-
After a back-and-forth over Twitter, making me sound like my focus was on her answers, the panelist and author direct-emailed the editor-in-chief of the offending blog expecting that it be corrected or removed. I will not share that email. Since the original blogger was already on Thanksgiving vacation, the choice was made to take it down.
-Attached is the zip file of what once was before the DMCA-equivalent take down.
-[archive] API Consumption at API STRAT_files.zip
-My point in writing this is that it is not okay to treat people like a piece of content. If you don't like being treated like a piece of ass, then don't treat industry professionals like they're personalities you can misquote.
-With minimal effort, you too can exercise control over marketing ignorance, both in your own business dealings and in others. It's as easy as starting shit on Twitter to help clarify misinformation right out in the great wide open.
diff --git a/_posts/2015-12-04-amazon-echo-and-the-future-of-digital-privacy.html b/_posts/2015-12-04-amazon-echo-and-the-future-of-digital-privacy.html deleted file mode 100644 index 32ca420..0000000 --- a/_posts/2015-12-04-amazon-echo-and-the-future-of-digital-privacy.html +++ /dev/null @@ -1,53 +0,0 @@ ---- -layout: post -title: Amazon Echo and the Future of Digital Privacy -date: 2015-12-04 18:03:46.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: [] -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '135' - dsq_thread_id: '4385950400' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2015/12/amazon-echo-and-the-future-of-digital-privacy/" ---- -Like every warm-blooded patriot of the Capitalism State, I own an Amazon Prime account. I vote, I pay taxes, I eat burgers, and I love the Boston Red Sox.
-There are no less than 4 Amazon-driven devices in my house: FireTV, Kindle Whitepaper, Kindle Fire HD, and a Kindle Dash near the dryer. The thing is, that’s not enough. For them…or for my family.
-
Alexa, Amazon’s newest home device, is about the size of a Quaker Oats tin, a compact but noticeable black cylinder that has a voice and listens to your every command. It’s a smart-speaker. It helps you play music when you want it, control lighting, remember appointments, and of course, place an order for something you’ve previously bought. It’s all very convenient, and it’s all very sinister too. Let me explain.
-Imagine if you’re a company as large, as powerful, and as consumer-driven as Amazon. With limited exception, you don’t produce much actual product. Primarily, you facilitate. You’re a services company. You are driven by how many people prefer to use you over something else. You do everything you can, including anti-trust tactics to make sure that consumers do use you, your services, and your products. The only reason you care about devices, as much trouble to produce and maintain as physical things are, is because those devices are a delivery mechanism for your services. Nothing sinister so far, right? Well, then…
-
When some other company wants to sell a product that competes with one of yours, like the pre-installed browser wars of the 90s, you wisely preclude them from selling their device in your marketplace (tough luck., Chromecast). You bundle your services and products in such a way that it’s a no-brainer to the average consumer to ‘add-on’ their way through your ecosystem, many of them completely unaware of any other alternatives to what you offer.
-
As a consumer, you develop a relationship to the company that gives your kids something educational to do every day, entertains you in the evening, delivers you your household items and groceries, hosts your websites, connects your devices, standardizes your co-workers’ software delivery pipeline, and generally make your life easier. But make no mistake, when theirs is the first online account that you update after receiving a replacement credit card, they've got you right where they want you.
-Yet still, all of that control over your consumer's lives isn't enough. You need more; you need to listen to every word their saying.
-So you create a cultural monopoly.
-Challenge: How do you get freedom-fighting citizens of a democracy such as the United States to voluntarily live in a surveillance state? Not through politics.
-Solution: Through technology. Invent devices that include microphones and/or cameras as a critical component to the purpose of the device.
-Result: People line up to sign away their liberty.
-
This should sound familiar. You already walk around with one on your person almost all day, every day. All major mobile phone platforms now have a feature for direct vocal interaction with its user. In many cases, this feature is enabled by default, which means that very easily your phone can be used as a surveillance device. Sadly, most people still think that their phone only listens to them when they ask it to. That is simply not the case, and I can prove it. Just hold up your Android with the feature turned on and, as if you were talking to a friend, say:
-“I think it’s shameful, when I typed in the wrong thing to Google, then all of the sudden I was presented with a bunch of dildos”.
-All Google hears is the last part of the sentence, and now it thinks you really like dildos. Lots of them. I'm not judging, it's just a thought experiment. It gets worse.
-
Imagine lawmakers who need to talk "in private" about how to deal with potential anti-trust activities that a company like Amazon is accused of committing. They’re talking over lunch, but one of them has a Fire Phone and this feature is turned on. It’s like Amazon has a way to live-stream private conversations, easily filtered and categorized by #Google because that’s how your device knows to listen up. We know other people have tried to do this nefariously. Imagine if you could do it legally?
-
Consider business owners talking privately about confidential stuff in their own home, with Alexa carefully listening to every word you and every other person in the room say. Your abode, your sanctuary, becomes a chapter from 1984, simply because you want an easy way to know what the afternoon weather will be like and you currently have dough on your hands
-
I find it strange that U.S. citizens get so angry when they are presented with facts on how the NSA has “secretly” listened to civilian conversations under the auspices of the Patriot Act and existing wiretapping procedures, but the same people line up at Apple stores for days when it’s something that brings them any marginal potential to simplify their lives. People will pass reforms before they even think about how traditional laws mean nothing to technology without formal transparency measures in place. They hastily scroll to the “ACCEPT” button at the bottom of a EULA that egregiously violates their digital human rights, and then are disappointed when the “convenience” factor isn’t up to their overblown expectations.
-By them I mean me. I grew up with Asimov in one ear and Philip Dick in the other. Judgement Day was inevitable. Zero-One, the Machine City, was impenetrable. So I have a healthy dose of skepticism about pervasive technology and artificial value.
-
But I also hope to see a world that works for me, not against me. I want my kids to grow up in an age where every new technology doesn’t automatically infer a forfeit of privacy and freedom. I want to know, not just feel, like better technology is a good thing, and I'm willing to pay for it, just not with my digital liberties. Me and my family are not royalty-free content, regardless of what our legal system lets tech startups get away with.
-
I think we can do it, but it will take an ethical shift in how tech companies maintain responsibility and transparency in their use of user data. Something like a report from an independent 3rd party audit that details exactly what the company is doing with data points they collect about their users, activity, and reselling that data. That would certainly light a fire under SaaS-hole startups and enterprises alike to not be creepy. I don’t think they’d like that thought. Amazon, Google, IBM, Facebook, Twitter…they’d all send their best lobbyists.
-How would you change this dynamic, the politics of digital privacy, and the inevitable impact of technology on your home life?
diff --git a/_posts/2016-01-09-new-years-resolution-dont-forget-the-little-things.html b/_posts/2016-01-09-new-years-resolution-dont-forget-the-little-things.html deleted file mode 100644 index a81a54e..0000000 --- a/_posts/2016-01-09-new-years-resolution-dont-forget-the-little-things.html +++ /dev/null @@ -1,38 +0,0 @@ ---- -layout: post -title: 'New Year''s Resolution: Don''t Forget the Little Things' -date: 2016-01-09 19:51:52.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: [] -tags: [] -meta: - _edit_last: '1' - dsq_thread_id: '4481445746' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2016/01/new-years-resolution-dont-forget-the-little-things/" ---- -Resolutions are stupid. Goals and a plan are much better IMO. My goal for 2016 is "don't forget the little things". For me, that means getting much better at JIRA.
-There are some days where I wonder how all of this works. For weeks now, "seasonal" sickness has been taking its toll on everyone, families, workplaces, suburban and metropolis, doesn't matter. Everyone has symptoms.
-A sample of my 5am train ride early last week, thoughts:
--
-
- zip your fly -
- kiss your son goodbye -
- bring home bag from work -
- collect homemade caramel corn jars -
- check the online gifts orders on the way home -
- (all the work things) -
- practice sanchin whenever possible without looking weird -
- buy more travel size tissues for backpack -
- queue up "thank you" holiday emails and tweets -
This is why we have tools like JIRA and Trello. Put it in there, try to get it right, and remember to check it frequently. There's too much involved in the plans to expect that large goals will be reached without something to keep it together.
-JIRA and task completion is my focus this week.
diff --git a/_posts/2016-01-11-gloucester-fire-housing-and-neighborhood-life.html b/_posts/2016-01-11-gloucester-fire-housing-and-neighborhood-life.html deleted file mode 100644 index 2a099e0..0000000 --- a/_posts/2016-01-11-gloucester-fire-housing-and-neighborhood-life.html +++ /dev/null @@ -1,35 +0,0 @@ ---- -layout: post -title: Gloucester Fire, Housing, and Neighborhood Life -date: 2016-01-11 08:50:42.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: [] -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '188' - dsq_thread_id: '4481411322' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2016/01/gloucester-fire-housing-and-neighborhood-life/" ---- -This morning, someone died in a fire in my neighborhood. My heart aches for them and theirs. There have been other fires, all of them terrible and sad too. There are many old houses in my town, and thought we don't know the cause of the fire yet, housing has been on my mind.
-I am a renter, been so for around 15 years in the area. I still have big questions:
--
-
- How the hell am I going to afford a house, ever? -
- What if I buy a house that's been flipped, and it burns down months later? -
- How safe is my family in a house that's older than the one in the news? -
- How will my neighbors handle the new August property tax hike? -
Tweets about gloucester fire hammond
-
I am a mess of feelings about this, grateful for those who serve on Gloucester Fire and Police duty. Grateful that it wasn't my space. Horribly sad for those who suffered and are suffering now. Proud that we have a hospital so close and so ready to respond. Scared it may happen to me one day.
-What I *do* know is that you never know what's going to happen next. Events like this are a reminder of how fragile, how precious, and how important life is, especially in a community that you love and trust. Not everyone is so lucky.
diff --git a/_posts/2016-01-14-how-do-you-test-the-internet-of-things.html b/_posts/2016-01-14-how-do-you-test-the-internet-of-things.html deleted file mode 100644 index a67a252..0000000 --- a/_posts/2016-01-14-how-do-you-test-the-internet-of-things.html +++ /dev/null @@ -1,55 +0,0 @@ ---- -layout: post -title: How do you test the Internet of Things? -date: 2016-01-14 22:23:22.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- API -- automation -- IoT -- quality -- testing -tags: [] -meta: - _edit_last: '1' - _oembed_99ed2ef0f9aa2f29b1c7078fbb86ebee: - _oembed_time_99ed2ef0f9aa2f29b1c7078fbb86ebee: '1452828017' - _thumbnail_id: '192' - dsq_thread_id: '4492433212' - _oembed_84dd5b75f1b1d604e54a458d3ebae3ec: - _oembed_time_84dd5b75f1b1d604e54a458d3ebae3ec: '1484590379' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2016/01/how-do-you-test-the-internet-of-things/" ---- -If we think traditional software is hard, just wait until all the ugly details of the physical world start to pollute our perfect digital platforms.
-[embed]https://www.youtube.com/watch?v=7odT95kmf_w[/embed]
-What is the IoT?
-The Internet of Things (IoT) is a global network of digital devices that exchange data with each other and cloud systems. I'm not Wikipedia, and I'm not a history book, so I'll just skip past some things in this definitions section.
-Where is the IoT?
-It's everywhere, not just in high-tech houses. Internet providers handing out new cable modems that cat as their own WiFi is just a new "backbone" for these devices to connect in over, in almost every urban neighborhood now.
-Enter the Mind of an IoT Tester
-How far back should we go? How long do you have? I'll keep it short: the simpler the system, the less there is to test. Now ponder the staggering complexity of the low-cost Raspberry Pi. Multiplied by the number of humans on Earth that like to tinker, educated or no, throw in some APIs and ubiquitous internet access for fun, and now we have a landscape, a view of the magnitude of possibility that the IoT represents. It's a huge amount of worry for me personally.
-Compositionality as a Design Constraint
-Good designers will often put constraints in their own way purposely to act as a sort of scaffolding for their traversal of a problem space. Only three colors, no synthetic materials, exactly 12 kilos, can I use it without tutorials, less materials. Sometimes the unyielding makes you yield in places you wouldn't otherwise, flex muscles you normally don't, reach farther.
-IoT puts compositionality right up in our faces, just like APIs, but with hardware and in ways that both very physical and personal. It forces us to consider how things will be combined in the wild. For testing, this is the nightmare scenario.
-Dr. Strangetest, or How I Learned to Stop Worrying and Accept the IoT
-The only way out of this conundrum is in the design. You need to design things to very discrete specifications and target very focused scenarios. It moves the matter of quality up a bit into the space of orchestration testing, which by definition is scenario based. Lots of little things are easy to prove working independent of each other, but once you do that, the next challenges lie in the realm of how you intend to use it. Therein lies both the known and unknown, the business cases and the business risks.
-If you code or build, find someone else to test it too
-As a developer, I can always pick up a device I just flashed with my new code, try it out, and prove that it works. Sort of. It sounds quick, but rarely is. There's lots of plugging and unplugging, uploading, waiting, debugging, and fiddling with things to get them to just work. I get sick of it all; I just want things to work. And when they finally *do* work, I move on quickly.
-If I'm the one building something to work a certain way, I have a sort of programming myopia, where I only always want it to work. Confirmation bias.
-What do experts say?
-I'm re-reading Code Complete by Steve McConnell, written more than 20 years ago now, eons in the digital age. Section 22.1:
-"Testing requires you to assume that you'll find errors in your code. If you assume you won't, you probably won't."
-"You must hope to find errors in your code. Such hope might feel like an unnatural act, but you should hope that it's you who find the errors and not someone else."
-True that, for code, for IoT devices, and for life.
diff --git a/_posts/2016-01-21-devops-burnout-and-the-search-for-the-holy-grail.html b/_posts/2016-01-21-devops-burnout-and-the-search-for-the-holy-grail.html deleted file mode 100644 index 3cef70d..0000000 --- a/_posts/2016-01-21-devops-burnout-and-the-search-for-the-holy-grail.html +++ /dev/null @@ -1,55 +0,0 @@ ---- -layout: post -title: DevOps, Burnout, and the Search for the Holy Grail -date: 2016-01-21 22:14:33.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- DevOps -- quality -- teams -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '204' - _oembed_6d6c82e7e115d76cb17490a3af35c812: "{{unknown}}" - _wp_old_slug: devops-burnout-and-continuous-improvement - dsq_thread_id: '4514395999' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2016/01/devops-burnout-and-the-search-for-the-holy-grail/" ---- -I'll be speaking at APIdays Melbourne about the technological equivalent of the holy grail, continuous deployment, and why maybe we should re-think certain dynamics coming from the push to "do DevOps", which like many good ideas is marred by poor implementations and shotty management.
2/2 Update: Things come up, shit happens, and I am incredibly bummed not to be able to be part of the crew at APIdays Melbourne this time around. However, priorities are priorities, and I'm not going to regret missing the 18 hour flight there and back.
-Grateful for the opportunity, hope this doesn't burn bridges, but sufficed to say I'll be there in spirit. Thinking of shipping a TelePresence bot and asking @switzerly to set it up for me. :)
-I'll still be looking to find a more local forum for this talk, hopefully at APIstrat.
-Of course, I'll be showing how to inject comprehensive testing into a pipeline of API design, build, deployment, and monitoring tools, but I'm a people person more than anything else, so germane to my presentation will be the topic of how "doing DevOps" affects us at a personal level too.
Humans are tool builders, not the other way around.
-Why are we talking about DevOps?
-I love the ideas coming from that space. Any time people work closer, tighter, better together, I'm down. But revenue doesn't care about you or me, and the impetus behind most practical implementations of continuous delivery are indeed revenue, over-trumped expectations from the business on IT as their main blocker rather than proper decision making.
-Often the result of forcing unprepared teams to "do DevOps": #burnout
-In November at APIstrat Austin I stood up and said that teams are more important to get right than the software they produce, though they're both very important. People produce software. If the people are buggy (i.e. bad team dynamics), you will see that in their product.
-At the company kick-off last week, I sat in the front row as a panel of exec-level customers validated that the immense pressure to release software faster than ever before is real, is connected directly to revenue (loss not just gain), and is incredibly challenging due to people problems more than just technological ones.
-Business leaders looking to implement new paradigms on technical teams will also find it surprisingly hard to "do DevOps" if there are cultural or personal issues laying around like land mines. From my last job, I know this first-hand.
-I'm a Developer, but My Cape is at the Dry Cleaners
-15 years professionally and counting. Right now, I see that code written in an IDE isn't the only important factor to bringing excellent products to market. Code of conduct in teams, the responsibilities a business has to its employees, and how we treat each other along the way to building world-class software are just as important for a sustainable business model
-Sorry startups who "do DevOps" because it's cool, call me in 6 months if you still exist and want to talk for real. I would *love* that as a podcast interview episode.
-For now, like an underwhelming version of Clark Kent, I temporarily hang up my [developer] superhero cape, put on thick-rimmed glasses, and work a job in the big metropolis during the day. I am educating myself and rounding out my ideas on what it really takes to be in cutting edge technology. I surround myself with very driven, passionate, fun, and smart people to get better...at everything I can.
-I am expanding my understanding of how to bring about great technology beyond what an IDE can provide me. I work with people, code, and businesses.
-More reading:
--
-
- Agile Admin: What Is DevOps -
- Matt Falken: Burnout, Recovery, and Honesty -
- 4 Biggest Reasons for DevOps Failure -
- Saving Your Infrastructure from DevOps -
- Burnout: the Not-so Secret Dark Side of Ops [Puppet Labs] -
- VictorOps: Burnout in Tech -
- Why DevOps is burning out developers -
Have you ever thought about what "departments" really means? The word "department" starts with another word: "depart". Stop, think, continue reading.
-Technical Chief Officer's Dilemma: Departments and "Agency"
-Are you in a situation where you honestly need people who purposely segregate themselves into groups that start with a departure from each other, rather than a congregation of ideas, people, and purpose?
-If you are responsible for a technology "department", you are responsible for a "failure". #explain
-Consider a geometric line, the most efficient way to connect one point to another. If only people were that easy. Get enough of them together and you start having to group them into manageable departments. IT, Development, Operations, Finance, Sales, Marketing, Management. Business lines to make things easy, right?
-Departments are "Depart"-ments
-Wrong. Department f*ck screw things up. Drawing lines isn't a good thing unless if it's to connect people with each other. They distract people from the simple truth that businesses who succeed are filled with people who instinctually understand that they are all on the same path, together.
Consider a geometric shape, the triangle. A line plus one point, an important point, an entire dimension. What good does it do to add another point beyond that? A square? Another department? Finance? HR? Marketing? Why?
-I'm minimizing, I know wonderful, necessary in finance and human resource. Apologies to them, it's just to make a point.
-Only the Right Lines Need to Be Drawn
-People who work with very large organizations know this inside out. Enterprises, government agencies, financial institutions. Corporations. The more lines there are, the more overhead and lack of progress there is. Sure, there's stability, structure, fortitude; but the further we get away from connecting point A to point B in a straight line, the less efficient we are.
-Truly effective business starts with figuring out how to define things with the least number of lines. Communication, organization, collaboration all benefit from simplifying how many lines are drawn. #karma
-More reading:
--
-
- IT conflict in UK companies stunts growth in claims DevOps studies -
- What Is the Best Way to Make Developer and QA Relationships Healthy? -
- O'Reilly Velocity, Burnout in Tech -
For software developers, APIs are a really logical choice for delivering how something should work on multiple platforms, acting as a sort of a common "language" between projects, systems, and teams.
-Who really 'uses' an API?
-Developers are the real 'users' of APIs. Sure everyone is the downstream recipient of the value of an API, but most of that doesn't happen unless an API satisfies its first audience: developers. They need to understand how it works, how to incorporate it into what they're designing, often designing APIs themselves. When something's hard to understand on an API, they're the ones that feel the pain, not the end user of the app.
-You might stretch the definition of an API 'user' to encompass 'anyone who benefits from the API', but it's really the developers who are adopting the API, proliferating it to others, and ultimately orchestrating the user's interaction with your API, like digital gatekeepers. Benefactors and users are important to differentiate between as a business; one represents the volume of your business, the other represents your adoption curve.
-Developers love something that just makes sense to them, and a well-designed API along with great documentation the best 'UI' an API can have.
-
But APIs don't have a UI!
-APIs lacks a User Interface (UI) in the traditional sense, that is until you go back to your definition of who you mean by 'user' when you say 'User Interface'. If the 'user' is a developer that interacts with code and APIs as their primary language, then you leave the UI up to them, but there is still an 'interface' to your API as a product.
-Traditional 'UI' focuses on treating the 'user' component in terms of human sensory input and direct manipulation. APIs being purely digital products, they are primarily something you interact with conceptually and socially.
-The social element of APIs is huge. By definition, by building an API, you are describing functionality in terms of what 'your' system can do for something else. APIs infer interaction, collaboration, and inclusion.
-The role of UI and APIs in the User Experience
-User interface (UI) is heavy work in terms of software; people slave over the perfect font, color scheme, and behavioral appropriateness of UI, only to find that the project is hard to maintain, change, or integrate with afterwards. Instead of thinking about how things look, good designers also consider how things behave.
-User Experience (UX) design is a field that accommodates these considerations while also maintaining a keen eye on the consumer's actual use, psychological reasoning, and emotional reactions to how a technology works. Yes, we need awesome UI, but only as a result of thinking about the human experience driving the functionality of the software, and only after we get the technical premises correct for the business requirements.
-What Makes for a Great Developer Experience?
-Developer Experience (DX) is the UX of APIs, but what lessons in great UI design can we apply to DX? What elements of great DX are unique to this non-traditional view of UX/DX?
-If I had to boil it down, a minimum-viable API developer experience in order of importance consists of:
--
-
- Documentation, both human-readable and machine-readable
-Hand-crafted API documentation is adorable, but most developers expect an M2M format like Swagger these days. A wise man once said: "he who hand-codes what he could have generated automatically using M2M descriptor is not spending that time making his API better".
- - Minimize number of exceptions, not the exceptions themselves
-I get it. You're special. We're all special like snowflakes. Yay for us. Sad for users of your API. You may have some CRUD operational endpoints on your API, but more than likely you have domain-specific activities that require some explanation. Explain when you deviate from the most obvious, and try to keep things as simple as possible. Be sure to call out any exceptions to your own (internal consistency) rules like naming and type/structure differences between operations. And for the love of whatever, document why you decided to do so.
- - Test the new developer on-boarding process, constantly
-Proving that your DX is what you need it to be must be done regularly. Use hackathons, free account sign-up, metrics, and feedback to make sure that there are no broken elements of the on-boarding chain. If pricing is untenable, you might want to have the business owners focus on that for a bit instead of quarreling over design or feature specifics. Like a reliable disaster-recovery plan, a great DX only comes from putting your expectations through their paces.
- - Be personable, real, and transparent as possible
-Developers are people. Against the perception from non-technical people, they aren't antisocial, they just hate bullshit and like to be passionate about really technical stuff. If your documentation, website copy, and support process reflect genuine concern for helping them solve their problem (and of course you do that too), you'll beourtheir friend for life, or at least until you have a major outage.
-
Who Else Says Developer Experience is Essential to A Great API?
-If you ask Jesse Noller, you'll get an earful about documentation and transparency, and I agree. Documentation is the first thing a developer will reach for when working with your API. That doesn't mean your efforts in DX should end there. You should also consider how other aspects like performance, authorization, and pricing play in to a developer's decision to implement and evangelize use of your API over another throughout their team and career.
-[embed]https://www.youtube.com/watch?v=-vZ_E1OO_PY[/embed]
-More reading:
--
-
- HackCU: An Example of Student Leadership -
- Why API Developer Experience Matters More Than Ever? -
- Pamela Fox at APIStrat: Developer Experience Matters[video] -
- Building Effective API Programs: Developer Experience (DX) -
This chick I know, I interviewed her last week for my upcoming podcast debut. She's phenomenal in a way that makes me so proud, grateful and humbled all at the same time. Sufficed to say, I found a place to put one of the ideas I had been holding on to ever since I started going to the Women In Tech group at my work:
-"Practical Tips to Help White Dudes Help Out"
-Trust me, I'm a subject matter expert on this. I'm so white, I made the vanilla ice cream we had for dessert tonight say "daaaaammn!" (then I ate it up). And though I'm not into sports and don't have lots of chest hair, I am definitely dude.
-I'm the kind of dude that wants to help. I have a son and daughter and I want the world to be a less sexist, less broken place by the time they are out in it. It's that attitude I look for in others, not the other things that differ between us.
-Yet it still remains, there are huge imbalances in society no matter what angle you look at things from. Good intention matters a little, but action in the face of injustice matters a whole lot more. It's what you do that defines you to others most because only a few people in life will ever spend the time to see past that.
-What Can We Do to Help?
-The difficulty with things like gender inequality, under-representation, privilege, and inclusion is that they're too nebulous/vague/ethereal for the side of the crowd that can/should/doesn't do something about it, namely white dudes, to take action on a daily basis. It's not that we're white or that we're dudes, but for whatever reason, many dudes just need practical instructions, marching orders, or technical requirements to move from well-meaning to noticeably effective.
-So I wrote some specific tips down. Their implementation might differ from person to person, so I wrote them in their most generic form:
-
Since they aren't exactly marching orders, more fortune cookie mnemonics, I'll put down some examples. They apply to all people, not just dudes to women, white people to everyone else, they apply to people who want to help other people. For the sake of this article, I'll write it as instructions to my fellow white dudes:
-1. Step up by being willing to step aside
-Instead of offering your own idea, ask a co-worker for her opinion first. This works best if you do it once or twice casually on a personal basis before doing it in a group or meeting.
-Doing so privately before hand can establish trust and help you understand if it's appropriate to do so in a group setting like a meeting, so that you don't accidentally put them on the spot.
-If you do successfully help to elevate someone else in a group, congratulations you're using your white dude privilege properly, that's why you feel good.
-2. Invert the situation in your head
-When people address a group as "guys, guys", think about what it would be like if you were in a group and someone addressed you as "ladies, ladies". It's a trite example, I know, figure of speech, but ask yourself: why is there even a gender associated with that figure of speech? #culture
-When was the last time you heard the words "aw, it's so great to see a man programmer, really brings some diversity of thought into the group"? or "really? you like beer? are you sure you don't want some wine or a fruity drink?"
-Gender/racial/sexual bias is baked in to _every_ aspect of American life, so there should be plenty of opportunities to invert the situation and see how subjugating it would feel to be on the other side of things.
-3. Learn where the gaps are around you
-Be willing to ask your human resources department to provide you statistics of gender, race, and ethnicity in your organization. Look around at how many black dudes or women are in your group? How about people from outside your background? If they say no, ask why? You can't be fired for asking about this stuff. If you are, then be glad! You're no longer working at the wrong place to work.
-4. Don't chalk things up to a stereotype
-Please, white dudes, please do not in your head justify the actions that a woman is taking with the fact that she is a woman. Do not think that he's thinking that way because he was raised in the ghetto (a.k.a. where all ignorant white dudes think black people come from). And for the love of whatever, please do not justify your homophobia by saying "so long as he doesn't try to hit on me, I'm cool with it".
Stereotypes limit people to presumptions you have about them regardless of their actions, which are the one thing we all control about ourselves. Reduce how someone chooses to put themselves out into the world, and you reduce your capacity to see clearly, to respect, to love, and to be loved.
-5. Listen; be more interested than interesting
-I will never reach a point where I can't get better at listening. I'm terrible at it today, I hope to suck at it less tomorrow.
-The more you listen (awareness), the more you maximize your opportunities. It's that simple. Action without listening is ignorance.
-The practical way to do this is to write "STFU" on your hand, on your notepad or tablet before a meeting, or picture everyone in the room having it tattooed to their foreheads.
-When you actively listen to someone, you express interest in them. People like to feel interesting, just like you, and giving that feeling to them as a gift is not a complicated or expensive affair. Both parties win in the end.
-6. Find a liaison, socialize, and invite
-It's intimidating to visit someone else's group or circle. The easiest way to smooth that social gravel is to have someone native invite you and liaise between you and the group.
-This puts a responsibility on you to be inviting and socialize people not in your group. It also puts a responsibility on all of these groups to be inviting and look for opportunities to become a liaison too. Yes, I'm calling everyone out here.
-Women in tech, take the time to bring a white dude to group. Black people, there is so much I don't deserve, but the privilege I have you're welcome to it so long as you're my friend. We share friendship, we share privilege. That's one way to get things flowing in both directions.
-7. Don't let failure stop you from trying again
-All of these things will feel awkward, not just for white dudes, but for everyone involved. Creating something new doesn't come easy. Easy is comfortable. If you're going to be uncomfortable, let it be because of something worthwhile.
-Other people are worth it. Try again. Don't push it...if you're doing #5 well, you'll know when to back off. But don't let failure stop you from doing the right thing. If there are others doing the same, the effect of trying will multiply itself in time.
diff --git a/_posts/2016-03-29-big-data-no-context-big-problems.html b/_posts/2016-03-29-big-data-no-context-big-problems.html deleted file mode 100644 index 6f559f0..0000000 --- a/_posts/2016-03-29-big-data-no-context-big-problems.html +++ /dev/null @@ -1,45 +0,0 @@ ---- -layout: post -title: Big Data, No Context, Big Problems -date: 2016-03-29 20:30:34.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- big data -tags: [] -meta: - _edit_last: '1' - dsq_thread_id: '4704147785' - _thumbnail_id: '259' - _wp_old_slug: big-data-without-context-is-for-big-idiots -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2016/03/big-data-no-context-big-problems/" ---- -
Just look at this graph...not a good trend, right?
-How do we know?
-Graph(Big Data) - Context == Big Problem
-Graphs are visualizations of data optimized for a particular audience: people.
-We don't have to be idiots to need simplification. All we need is to be is modern digital citizens, overwhelmed with data, overburdened by it in many cases.
-Graphs follows certain principals of design, common expectations, crowd psychology. To illustrate growth in a left-to-right culture, you illustrate low values on the left and show them increasing on the right. Common x-plot values are time, stdev (standard deviation), and relevant ranges of independent variables.
-As an aside, I *LOVE Bell curves. Show me your Bell curves and I'll pay for dinner. Every. Time.
-Big Data == Insight - Context
-Going back to the overwhelming amount of data in our lives, with all this "big data", we still need context...framing around the target problem, which is where "big data" fails for me. It speaks to what it is, but not why. That's where design comes in.
-"Design" infers an audience, a "thing" being designed for that audience, and a conscious mapping of the needs of that audience to the features of the thing. Purposeful design is one step beyond that, extending context to a domain, effective use of the "thing".
-If you're in the business of big data, you're not an idiot. You are probably purposely designing context into that market every day, whatever it is you're doing. Just, every so often, ask yourself: "Am I providing value to someone?"
-Big Data + Context == Insight
-Here's a different example of the same values for both axes at the same scale:
-
What's interesting in this graph is how there was a dramatic spike at one point followed by a sustained average. The slight dip and hike towards the end is ambiguous in meaning unless you had that sustained average.
-Until now, I've purposely left out context for these graphs to make a point. Here it is for this one: someone got smart about PR in 2014, didn't fake anything, and is working hard to make sure people are happy.
-Graphs are worthless without context. Don't expect dashboards to save you unless you understand what the graphs really mean in context.
--
diff --git a/_posts/2016-04-16-the-cost-of-not-changing-things-up.html b/_posts/2016-04-16-the-cost-of-not-changing-things-up.html deleted file mode 100644 index 8ebf2fc..0000000 --- a/_posts/2016-04-16-the-cost-of-not-changing-things-up.html +++ /dev/null @@ -1,40 +0,0 @@ ---- -layout: post -title: The Cost of Not Changing Things Up -date: 2016-04-16 22:06:01.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- open source -tags: [] -meta: - _edit_last: '1' - _oembed_9baacc459bfd1337ed3d5ba1f021334a: "{{unknown}}" - _thumbnail_id: '268' - dsq_thread_id: '4753047119' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2016/04/the-cost-of-not-changing-things-up/" ---- -
This week, I realized that for years, I've been looking at life changes in terms of cost. How much will it cost to move, how much will it cost if I quit, how much will it cost if I fail.
-Nothing of worth comes on a silver spoon
-Despite warnings and signs, friends and teachers, and despite being part of moments in spacetime I would gladly revisit, my aha moments always necessarily originate internally, most often when I put myself in challenging situations. I always want to know what's in the pita.
-While waiting for an officer to write me a bogus citation today, it hit me that much as in the same way I measure things from both sunk vs. opportunity cost perspectives in business, I forgot to do that with my home and personal life. The cost of what I'm doing in one place is more than what it costs me emotionally, physically, and mentally; it's costing me true happiness that I'm leaving on the table.
-Forget silver, there is no spoon
-I love this thing. It's not my family (of course I love them), it's literally a thing. I have fallen in love with an open source thing. That passion has caused me to ignore the bigger picture. And who the fuck loves a thing over themselves?
-When someone or something is tied to a prior transformation, it's hard to give it up. It's an edifice, a token. It's also just a thing. And since this particular thing is open source, it's free (for now at least) so at least there's no sunk cost for me to keep using it.
-There may be no spoon, how about a knife
-I feel like a snake during ecdysis, or maybe a pile of phoenix ash. Either way, change is coming, and bitches, watch what happens between here and Denver. I have something no one is expecting. I will cut through so much wrapping paper that the present inside will be a fun little aftershow.
-Semi-related reading:
--
-
- CTO vs. CIO: How many tech “corners” do you really need? -
- 7 Practical Tips for Inclusion -
- DevOps, Burnout, and the Search for the Holy Grail -
It was about 7 months ago I started to feel myself drawing a line about how I use the words "open source". 7 months. Certainly in my copy, my writing, and my ideas for future projects (not just programming projects), these words infer meaning beyond what most people think about when they use them.
-Originally, open source software was just a bunch of scientists sharing useful stuff. Now its a method of brand extension. How far we've all come. It used to mean the freedom to do what one thought was worth doing. Now it seems more synonymous with another word: shackles.
-So I want to put it down here, a point in time, where people can see my progression. To call something open source, I expect that:
--
-
- it's IP/source is available freely without signup -
- it is moderated by the community of its own contributors, not one entity -
- it is maintained separate from other commercial product/service lifecycles -
- does not share the identity of its benefactors consent -
- does not contain EULAs that non-technical and non-legal persons can understand -
I have no problem with companies who want to use "open source" to their brand advantage, but to not respect these few core principals is to be 'half in' and very transparent in its intent.
-https://www.youtube.com/watch?v=jw8K460vx1c
-Other reading:
- diff --git a/_posts/2016-04-25-kindness-what-makes-a-great-technical-team.html b/_posts/2016-04-25-kindness-what-makes-a-great-technical-team.html deleted file mode 100644 index d6931e6..0000000 --- a/_posts/2016-04-25-kindness-what-makes-a-great-technical-team.html +++ /dev/null @@ -1,40 +0,0 @@ ---- -layout: post -title: 'Kindness: What Makes a Great Technical Team' -date: 2016-04-25 20:49:48.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- developer -- teams -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '287' - dsq_thread_id: '4776851385' - _wp_old_slug: kindness-what-makes-a-great-technical-team-17 -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2016/04/kindness-what-makes-a-great-technical-team/" ---- -How do you quantify what makes a great working environment, a good team, and work worth doing? An important piece for me is kindness.
-Kindness is Human
-At the risk of speaking to some stereotypes here, kindness is incredibly important in teams where members often lack patience and social skills. It doesn't have to hinder honesty, expedience, or effectiveness. In fact, showing a small amount of kindness in your daily interactions reminds the people you often work most closely with on otherwise dry, tactical topics that there's more to work than just code or tech. It reminds people that both you and they are human, not software-delivering carbon units.
-Kindness is Engaging
-When you show a little kindness, maybe a bit of sympathy about how frustrating a specific framework or project is to the team, it allows people the opportunity to open up if they want to. By sharing a short, discrete moment where you've felt the same excitement or frustration for a technology, you'd be surprised how often it elicits candid conversations that you'd otherwise not have during normal sprints or reviews. It gives people the cue that it's okay to be themselves a little.
-Kindness is Strategic
-Looking for a promotion? Counter to kissing ass with a boss, expressing a little kindness to everyone you work with signals to intelligent management that you pay attention to social cues and understand how to play well with others.
-Again, the level of kindness you show shouldn't detract from your efficiency or ability to be straightforward at work; it should enhance your efficiency by proactively smoothing the interactions you have with co-workers, partners, and other non-technical folks.
-Kindness is Admirable
-Everyone has bad days. It's how we deal with them that signals to others how professional, reasonable, and capable you are when you get down to business. And let's be honest...technology is a revolving door. The relationships you make at one organization can indefinitely benefit future positions you may hold. Just as honesty and effectiveness are qualities people remember, being unkind never gets you on the list of people that others want to carry with them to the next gig.
-More reading:
- diff --git a/_posts/2016-05-30-gluecon-2016-melody-meckfessel-on-serverless-technology.html b/_posts/2016-05-30-gluecon-2016-melody-meckfessel-on-serverless-technology.html deleted file mode 100644 index ead5472..0000000 --- a/_posts/2016-05-30-gluecon-2016-melody-meckfessel-on-serverless-technology.html +++ /dev/null @@ -1,62 +0,0 @@ ---- -layout: post -title: 'Gluecon 2016: Melody Meckfessel on Serverless Technology' -date: 2016-05-30 11:20:56.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- API -- automation -- developer -- DevOps -- Gluecon -- open source -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '326' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2016/05/gluecon-2016-melody-meckfessel-on-serverless-technology/" ---- -I had the opportunity to attend Melody Meckfessel's presentation at Gluecon 2016 last week. As someone with years of experience at Google, when she speak, smart people listen.
-[paraphrased from notes]
-We should all be optimistic about open source and serverless technologies. Kubernetes, an example of how to evolve your service, a means to an end, housing processes internally, making management easier for developers to deploy and scale horizontally, to update gracefully without causing downtime. One of the themes in the future of cloud-based computing is that is increasingly open source which makes it easier for developers to influence and contribute the software stack as it's evolving. Our focus is switching to managing applications and not machines.
--"...the future of cloud-based computing...is increasingly open source which makes it easier for developers to influence and contribute the software stack as it's evolving."
-(photo credit: @rockchick322004)
Serverless means caring about the code, embedded in all cloud platforms. In App Engine, you run code snippets and only get charged for your app or service as it scales up or down. With container engine, you run containers but don't specify the machine it runs on. In the future, we're not going to be thinking or talking about VMs. What does it mean for us developers? It will enable us to compose and create our software dynamically. As one of the most creative and competitive markets, software puts us in the same room and presents us with the same challenge: how to make things better, faster.
-Melody tells a story about a project she was working on where they were using a new piece of hardware. They were having trouble scaling during peak load, holiday traffic being really important to their particular customer. They were constantly scrambling and for the first 6-9 months, she spent her time on primarily DevOps related work. This was frustrating because she wanted to work on features, seeing the potential of what she could do, and she could never quite get to that.
-As developers, we are constantly raising the bar on the tools we use to release, and correspondingly our users' expectations increase. Also as developers, we have the ability to create magic for users, predicting what they want, putting things in context, and launching the next great engaging thing. In a serverless future, developers will be focused on code, which means that PaaS (platform as a service) will have to evolve to integrate the components it's not doing today. To do that, developers will need to shift Ops work from themselves more to cloud providers, being able to pick and choose different providers. Developers come to a platform with a specific language that they're productive in, so this is a place where PaaS providers will have to support a variety of runtimes and interpreters. App Engine is the hint and glimmer of creating an entirely new developer experience.
--"In a serverless future, developers will be focused on code...will need to shift Ops work from themselves more to cloud providers."
What does a serverless future mean for the developer experience?
-We will spend more of our time coding and have more and more choice. Right now, we're in a world where we still have to deal with hardware, but as that changes, we'll spend more of our time building value in our products and services and leave the infrastructure costs to providers and the business. If we want to do that, developers need to make it very easy to integrate with machine learning frameworks and provide analytics. Again, to do this, we need to free up our time: hence NoOps. If we're spending all out time on ops, we're not going to get to a world where we can customize and tailor our applications for users in the ways they want and expect. This is why NoOps is going to continue to go mainstream.
--If we're spending all out time on ops, we're not going to get to a world where we can customize and tailor our applications for users in the ways they want and expect.
Think about if you're a startup. If you're writing code snippets, you may not need to think about storage. You don't need to worry about security because the cloud providers will take care of that for you. Some wifi and laptops, and that's your infrastructure. You'll have machine-oriented frameworks to allow that app to be really amazing. This idea that you can go from prototype to production-ready to then scale it to the whole planet. Examples of really disruptive technology because they were enabled to do so by cloud resources.
-To get there, we're going to have to automate away the configuration and management and the overhead. We still have to do this automation. We have work to do there.
-Multi-cloud in NoOps is a reality.
-People are using mixed and multiple levels of on-prem IT and hybrid cloud; the problem is that the operational tools are siloed in this model though. This makes it very hard to operate these services. As developers, we should expect unifying management...and we should get it. Kubernetes is an example, just another way for teams to deploy faster. Interestingly, we're seeing enterprises use Docker and Kubernetes on-prem; the effect of this is that it will make their apps and services cloud-ready. When they are faced with having to migrate to the cloud, it will be easy to do it.
--"...we're seeing enterprises use Docker and Kubernetes on-prem...it will make their apps and services cloud-ready"
Once you're deployed like this though across multiple clouds and service providers, you'll need to have an easy way to collect logs and perform analysis; StackDriver is an example of this. It's this single-pane-of-glass view that enables developers to find and fix issues fast, to see what's happening with workloads, and to monitor more effectively. Debugging and diagnosing isn't going to go away, but with better services like Spinnaker and App Engine, we'll be able to manage it more effectively.
-Spinnaker providers continuous delivery functionality, tractability throughout our deployment process, it's open source. As a collaboration with Netflix, Melody's team worked hard on this. It was a case of being more open about the software stack we're using and multiple companies coming together. We don't all need to go build our own thing, especially when we share so many common problems.
-
Debugging and diagnosis
-The problem is that there's all this information across all these sources and it's hard to make sense of it. It's usually in a time-critical situation and we have to push out a quick fix as soon as we can. Part of streamlining that developer and debugging experience in the future cloud is having the tools at our disposal. These are things like being able to trace through a complex application for a performance issue, going back through the last 3 releases and see where the slow-down started, or production debuggers where you can inspect your service that's running in the cloud and go to any point in the code to look through variable and see what's happening. Error reporting as well needs to be easier, as errors can be spread across multiple logs, it's hard to see the impact of the errors you're seeing. Errors aren't going away, so we need to be able to handle them effectively. We want to reduce the effort it takes to resolve these errors, we want to speed up the iteration cycles for finding and fixing the errors, and provide better system transparency.
--"The faster we can bring the insight from these motions back in to our source code and hence optimize, those are all benefits that we are passing through to users, making us all happier."
In the NoOps, serverless future, not everything will be open source, but we will expect that 3rd party providers should work well with all the cloud platforms. Meledy's team is currently working on an open source version of their build system called Basil. Right after their work on Spinnaker in collaboration with Netflix, they're looking for other opportunities to work with others on open source tools.
-What does the world look like 5-10 years from now?
-We're not talking about VMs anymore. We're benefiting from accelerated hardware development. We're seeing integration of data into apps and compute so that we can have better insight into how to make applications better. It's easier for machine learning to be embedded in apps so that developers can create that magical software that users expect. We don't have to talk about DevOps anymore. We will have more time to innovate. In an open cloud, you have more opportunity to influence and contribute to how things evolve. Like thinking back on how there used to be pay phones and there aren't anymore, the future of development is wide open, no one knows what it will look like. But contributions towards the future cloud are a big part of that future and everyone's invited.
-More resources:
--
-
- [video] - -
- The Woman at the Heart of Everything Google Does -
- Fortune: Girls Should Play with Tech-Savvy Toys -
[caption id="attachment_366" align="alignright" width="150"]
Meme == fun! Make one yourself! Tag #SummerOfSelenium[/caption]
I rarely go to the beach. When I do, I like to do what I like to do: surprise, tech stuff. We all relax in different ways, no judgement here. We also all need to work flexibly, so I'm also busy with a new co-working space for my locals.
-In back and forth with a colleague of mine, we started to talk about strange places where we find ourselves writing Selenium scripts. All sorts of weird things have been happening in mobile this summer, Pokemon Go (which I call pokenomics) for example, and Eran was busy creating a cross-platform test script for his upcoming webinar that tests their download and installation process.
-At work, we're doing this #SummerOfSelenium thing, and I thought that it would be cool to start a meme competition themed around summer-time automated testing from strange places. Fun times, or at least a distraction from the 7th circle of XPath hell that we're still regularly subjected to via test frameworks.
-If you want to build your own meme, use the following links...
-
Reply to my tweet with your image and I'll figure out a way to get our marketing team to send you some schwag. ;D
-Mine was:
-
Then Eran responded with:
-
We think we're funny at least.
--
Side-note: co-working spaces are really important. As higher-paid urban jobs overtake local employment in technical careers, we need to respond to the demand for work-life balance and encourage businesses to do the same. Co-working spaces create economic stickiness and foster creativity through social engagement. My thoughts + a local survey are here, in case you want to learn more. A research area of mine and one I'll be speaking on in the next year.
diff --git a/_posts/2016-08-03-why-responsive-web-developers-must-learn-selenium.html b/_posts/2016-08-03-why-responsive-web-developers-must-learn-selenium.html deleted file mode 100644 index de039d5..0000000 --- a/_posts/2016-08-03-why-responsive-web-developers-must-learn-selenium.html +++ /dev/null @@ -1,70 +0,0 @@ ---- -layout: post -title: Why responsive web developers must learn Selenium -date: 2016-08-03 21:29:06.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- RWD -- Selenium -tags: [] -meta: - _oembed_time_dfe79c0fc9ebb6611bbe0d77ad7ee9af: '1470229878' - _edit_last: '1' - _oembed_dfe79c0fc9ebb6611bbe0d77ad7ee9af: 'Full-stack - developers' - _thumbnail_id: '385' - dsq_thread_id: '5039821568' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2016/08/why-responsive-web-developers-must-learn-selenium/" ---- -
Developers need fluency in technologies that the team uses for UI testing. Similarly, testing has become more a programmer's job than a point-n-click task. As the world's most widely-adopted web UI testing framework, Selenium is a must-know technology for responsive web development teams in order to maintain the pace of continuous delivery.
-Mostly Working, Somewhat Broken 'Builds'
-I use the term 'build' loosely here when it comes to web apps, since it's often more like packaging (particularly with Docker containers), but the idea is the same. You have code in a repo, that gets sucked in by your CI system, built/packaged), tested/validated, then deployed somewhere (temp stack or whatnot).
-If the build succeeds but tests fail, you need to quickly assess if it's the test's fault or the new build. The only way to do that quickly is to maintain proficiency in technologies and practices used to validate the build.
--Code in production is where money is made; broken code doesn't make any money, it just makes people move on. Everyone is responsible for the result.
We have a QA person. That's their job.
-Not so fast. In continuous delivery teams, developers don't have the luxury of leaving it to someone else. When build verification tests (BVT) fail, a developer needs to track down the source of the problem which typically means parsing the test failure logs, referring to the code and the context/data used for the test, then making some adjustment and re-running the tests.
-Though your test engineer can fix the test, you still have to wait for a developer to fix problems in code. Why not make them the same person?
-Of course I'm not suggesting that every developer write ALL their own UI/UX tests, there's a division of labor and skills match decision here which each team needs to make for themselves. I also thing in larger teams it's important to have separation of concern between those who create and those who validate.
-Code in production is where money is made; broken code doesn't make any money, it just makes people move on. Everyone is responsible for the result.
-How can web developers find time for UI testing?
-The answer is simple: do less. That's right, prioritize work. Include a simple test or two in your estimation for a feature. Put another way, do more of the right things.
-You may get push-back for this. Explain that you're purposely improving your ability to turn things around faster and become more transparent about how long something really takes. Product owners and QA will get it.
-Now that you have a sliver of breathing room, learn how to write a basic Selenium script over what you just created. Now you have a deliverable that can be included in CI, act as the basis for additional validation like load testing and production monitoring, and you look very good doing it.
-You can do it!
-Obviously, you can go to the main site, but a quick way for those with Node already installed is through WebDriver:
-require('webdriverio')
- .remote({desiredCapabilities:{browserName: 'chrome'}})
- .init()
- .url('http://www.github.com')
- .setValue('.header-search-input','perfectomobile')
- .submitForm('.js-site-search-form')
- .getText('.codesearch-results * h3').then(function(value) {
- assert(value[0].indexOf('repository results')>-1);
- })
- .end();
-
-For those who have never worked with Selenium before, start with this tutorial. Then go through the above.
-Even if you aren't a Node.js fan, it's an easy to grasp the concepts. If you want to go the Java or C# route, there are plenty of tutorials on that too.
-More reading:
--
-
- StackOverflow: Is it realistic to hire someone who is a developer and a tester? -
- Mike Loukides on Full-Stack Developers - -
diff --git a/_posts/2016-08-05-dear-sauron-i-will-not-be-returning-to-mordor.html b/_posts/2016-08-05-dear-sauron-i-will-not-be-returning-to-mordor.html deleted file mode 100644 index c730778..0000000 --- a/_posts/2016-08-05-dear-sauron-i-will-not-be-returning-to-mordor.html +++ /dev/null @@ -1,32 +0,0 @@ ---- -layout: post -title: Dear Sauron, I will not be returning to Mordor -date: 2016-08-05 22:29:16.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: [] -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '282' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2016/08/dear-sauron-i-will-not-be-returning-to-mordor/" ---- -
[originally written 4/22/2015]
-Just to be clear, I'm not talking about J.R. Tolkien.
-I have been to the Mountain. I have felt the Ashes on my brow. I have seen what a skilled Practitioner with a broken soul does to people around her. There's a difference between assertive and asshole. There's skilled and there's skewed.
-I can't imagine what it's like to be an immigrant. I mean, I've talked to my Russian grandmother (rest her soul) about Baba, her mother. I remember pictures of her and only understand very little about what it's like to be an stranger to everyone around me. I have friends who went through this, but have never been one myself, except for the past 14 months of life.
-I ignore homeless people every day when I go into the city. I disregard their need every time I don't give them the change from my pocket. One time, I saw a Vietnam veteran with a cardboard sign asking for new glasses. The next day after hitting an ATM, I looked for him in on the way back in. I never found him. I should have overcome my ego problems and just eaten the $4 ATM fee right then and there, gone back, and walked with him to the nearest LensCrafters. Do you know how sad it feels to have hundreds of dollars of cash in your pocket for someone who you'll never see again? You won't if you can't.
-Remorse, empathy, kindness...these are character traits of someone I'm proud to be. Domination, cynicism, rudeness...they are transparent to people who care. After all this time, I still do; but broken is broken, and as much of an engineer as I am, some things can't be fixed. Some things aren't worth the effort.
-Additional Reading:
- diff --git a/_posts/2016-08-10-you-must-be-this-high-to-ride-the-continuous-bandwagon.html b/_posts/2016-08-10-you-must-be-this-high-to-ride-the-continuous-bandwagon.html deleted file mode 100644 index a959ead..0000000 --- a/_posts/2016-08-10-you-must-be-this-high-to-ride-the-continuous-bandwagon.html +++ /dev/null @@ -1,57 +0,0 @@ ---- -layout: post -title: You Must Be This High to Ride the Continuous Bandwagon -date: 2016-08-10 21:23:50.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- continuous deployment -- DevOps -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '394' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2016/08/you-must-be-this-high-to-ride-the-continuous-bandwagon/" ---- -There’s a lot of hype when it comes to continuous deployment (CD). The fact is that in large organizations, adopting CD takes changes to process, responsibilities, and culture (both technical and management). The right skills really help, but more often the determining factor to success is having the right attitude and vision across the whole team.
-[caption id="attachment_395" align="alignnone" width="600"]
(image via Yassal Sundman)[/caption]
At a carnival, you may have seen a sign that says “you must be this tall to ride”, an indication that the attraction is designed in such a way that it is dangerous to ride for those who don’t meet the specification. Similarly, continuous deployment sets the bar of requirement high, and some teams or products aren’t set up to immediately fit into this new methodology.
-Mobile Continuous Delivery Requires Micro-climates
-Mobile apps go through a validation process in an app store or marketplace before being generally available to customers, so product feedback loops take a hit in delay to market response to the app update. Mobile apps typically rely on back-end infrastructure which may require synchronous roll out of both front-end app and server-side components such as APIs and database schema. This is not trivial and for apps with thousands to millions of users.
-Because of this delay, there’s huge emphasis on getting mobile app changes right before submitting them for review. Internal and beta testing platforms like TestFlight for iOS and HockeyApp for Android become vital to a successful app roll out and update strategy. For organizations that are used to 3 month release cycles and who control their whole stack, being prepared to release perfection every week requires a completely different mentality, often a completely different team too.
-This is what I call product 'micro-climates', an ecosystem of people, processes, tools, and work that evolves independent of the larger organization. Mobile and API teams are perfect examples. A product needs to go at it's own pace, accelerate and improve based on its own target audience. Only when organizations align product teams to business goals does this really take hold and become effective.
-Prove Your Success, Aim for a Shared Vision
-I've never seen a Fortune 500 organically evolve to CD without buy-in from a C-level or at least VP. A single group can implement it, but will ultimately run into cultural challenges outside their group (like IT and infrastructure) unless they have the support of someone who controls both groups.
-If you're trying to move in that direction but are hitting barriers outside your team, you've may have bitten off too much for now, and need buy-in from above (i.e. an executive sponsor). For that, you need:
--
-
- Proof that what you're doing is actually improving your velocity
-'DevOps' is a buzzword, but metrics that show how doing kanban/scrum with both teams in the room every day actually matters. If you aren't already capturing these metrics, I'd suggest you start. The point is to have quantifiable, objective measures that undeniably show success.
- - How your success maps to your executive sponsor's goals
-An executive often balances potential opportunities with opportunity costs. If you're changing process, what's the risk to your actual project? How can this be replicated to other teams? What's at risk if you don't do this? Why are other competitors doing it this way too? What strategic objectives does this change enable (i.e. faster releases == competitive advantage)? Take a few moments to think about what your sponsor is measured on, and map your goals to theirs.
- - A clear plan and schedule, not just a bunch of activity
-Adding one or two process improvements is one thing, that's actually our responsibility anyway, but to move to a model like continuous delivery/deployment you need a plan that includes objectives, strategy, and then tactics. For instance: --
-
- Objective: meet demand for new features, obtain competitive advantage in market -
- Strategy: streamline the delivery process to achieve 1-2 week release cycles -
- Tactics:
-
-
-
- Continuous integration of code, multiple commits per developer per day -
- Minimum 80% automated test coverage -
- Test coverage over 5 key platforms and 3 geographic markets -
- Automated security reviews before each release (i.e. like this) -
- Tractability of code changes to production user impact metrics -
-
-
If you've been bitten by the CD bug, it's more than just an itch to scratch. It takes some concerted effort, particularly in large organizations, but don't let that hinder you. Get your own team on board, find your velocity metrics, link your proposal to executive goals, get that sponsor, and commit to an implementation plan. Others have done it, and so can you.
diff --git a/_posts/2016-09-03-why-espresso-unit-vs-ui-testing.html b/_posts/2016-09-03-why-espresso-unit-vs-ui-testing.html deleted file mode 100644 index 515e9a8..0000000 --- a/_posts/2016-09-03-why-espresso-unit-vs-ui-testing.html +++ /dev/null @@ -1,43 +0,0 @@ ---- -layout: post -title: 'Why Espresso: Unit vs. UI testing' -date: 2016-09-03 23:05:24.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- Android -- Espresso -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '415' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2016/09/why-espresso-unit-vs-ui-testing/" ---- -This article differentiates unit tests, such as those written for jUnit, from UI tests in Espresso through both purpose and technical value.
-What is Espresso?
-Espresso is an automated UI testing framework for Android apps. They are scripts written in Java that simulate interactions with the app while it is running, either in an emulated environment or on a physical device.
-Espresso tests are "instrumented", which means that internal workings (context) about the app such as object names, runtime variables, and other symbolic information is made available to the tests. Using Android Debug Bridge (ADB) to provide runtime feedback between tools like Android Studio and the app as test activities are executed on the target device.
-How is Espresso different than Unit Testing?
-Unit tests focus on small portions of code (i.e. class-level methods) and provide basic validation that code is working as expected. Espresso tests provide basic validation that the UI is working as expected.
-Early feedback from lots of tiny unit tests on each build help developers know when they just "broke" something by changing some other portion of code. Early means often, and often means that speed and reliability of these tests are crucial.
-To that end, unit tests typically are hermetic and rely on stubs/mocks to stand in for dependencies. Antithetically, Espresso UI tests work through platform API which requires a runtime and device capabilities that are not faked. This provides more realistic feedback on code that might work at the unit level, but fails when chained together or during basic usability validations.
-When should we write Unit Tests?
-Always. You can figure out for yourself what total percent of lines of code are covered by unit tests, but this is a battle against low-level technical debt. How often do you want to be surprised when a change that seemingly had nothing to do with one piece of code breaks because expectations over what that code does weren't spelled out in a way that could be exercised regularly?
-That's really what validation testing boils down to: are we communicating basic expectations about the things we're about to ship? Unit is just at a very low level, but the same applies at the UI workflow level too.
-When should we write UI tests?
-Whenever you have a UI...that's pretty obvious, right? People use an app, they don't call your class methods in isolation with static data on emulators. Eventually you have to get real: simulate clicks, drags, gestures, network conditions, and platform upgrades because that's how real people are using your user interface (a.k.a. your app).
-How much UI testing you do is up to you, but it boils down to time cost. UI tests are often more complicated to write, though as we see with Espresso, a developer-focused syntax and fast execution speed goes a long way to reducing cultural friction to writing UI tests as part of "development complete".
-Sideline: API teams, you don't get off so easily either. Your developer experience is your UI, the patterns by which your users call your endpoints, designed or otherwise, are equivalent to workflows. Equivalent to UI testing for app developers, holistic API tests that simulate known trends and expectations on your API will help you isolate breaking changes earlier and faster in your build cycles, it's that simple.
- diff --git a/_posts/2016-09-20-android-studio-how-to-find-the-package-name-from-apk.html b/_posts/2016-09-20-android-studio-how-to-find-the-package-name-from-apk.html deleted file mode 100644 index c70f63a..0000000 --- a/_posts/2016-09-20-android-studio-how-to-find-the-package-name-from-apk.html +++ /dev/null @@ -1,45 +0,0 @@ ---- -layout: post -title: 'Android Studio: How to find the Package Name from APK' -date: 2016-09-20 14:23:49.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- Android -tags: [] -meta: - _edit_last: '1' - dsq_thread_id: '5162034532' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2016/09/android-studio-how-to-find-the-package-name-from-apk/" ---- -If you've been given an APK file but don't know the ApplicationId/Package name, you can use the 'aapt' (Android Asset Packaging Tool) to obtain this value. What you'll need:
--
-
- Your Android SDK install directory path
-
-
-
- Mac: /Users/yourusername/Library/Android/sdk -
- Win: C:\Program Files\Android\SDK -
- - Your APK file (example here) -
You'll have to navigate beyond the SDK folder to the 'build-tools' and then your version number (see below) to find the 'aapt' tool. From that directory, you can run:
-aapt dump badging <path to your apk>
-
This will dump the details of your app manifest, the first line of which is your package name (a.k.a. the ApplicationId in a build.gradle file).
-Additionally, you can get a complete listing of the manifest by using:
-aapt list -a <path to your apk>
-Why Do I need the the Application ID / Package Name?
-For various reasons, this identifier is important. For instance, debugging with the ADB command tools or launching an existing app requires this ID.
-In my case, I needed to know what to add to a Perfecto test for the Espresso Execute test step:
-
I have to add the ".test" suffix to the package name coming from aapt because I need to tell the Perfecto Espresso executor to run tests.
-diff --git a/_posts/2016-11-09-intercepting-espresso-intents-a-security-concern.html b/_posts/2016-11-09-intercepting-espresso-intents-a-security-concern.html deleted file mode 100644 index 2910b97..0000000 --- a/_posts/2016-11-09-intercepting-espresso-intents-a-security-concern.html +++ /dev/null @@ -1,33 +0,0 @@ ---- -layout: post -title: Intercepting Espresso Intents, a Security Concern? -date: 2016-11-09 19:16:21.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- security -tags: [] -meta: - _oembed_45a8ffabbfc1f138c2eb038cc085859d: "{{unknown}}" - _edit_last: '1' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2016/11/intercepting-espresso-intents-a-security-concern/" ---- -
Just came across this nugget after Googling for 30 seconds.
-https://github.com/intrications/intent-intercept
-Essentially, you can mine an app for the intents it signals to the outside world, then intercept, then re-inject them with your own modified data. Does this seem like a potential app vulnerability to you?
-More research must be done, but this smells like something I want to bring up in my Edges of Espresso talk at AnDevCon SF this month.
-Update 2016-11-11
-Found this, old, but good article supporting my concerns. Intentional Evil: A Pen Tester's Overview of Android Intents
-Also, I keep re-reading this one on IntentTestRule usage because it's so how my brain works. http://www.catehuston.com/blog/2016/04/28/testing-intents-on-android-like-stabbing-yourself-in-the-eye-with-a-blunt-implement/
-Some really great use cases for Facebook Connect login stubbing here:
-https://medium.com/@_rpiel/how-to-test-facebook-connect-with-espresso-8a1af3e38d50
diff --git a/_posts/2016-11-09-looking-forward-to-codestock-2017.html b/_posts/2016-11-09-looking-forward-to-codestock-2017.html deleted file mode 100644 index 98ba4f1..0000000 --- a/_posts/2016-11-09-looking-forward-to-codestock-2017.html +++ /dev/null @@ -1,42 +0,0 @@ ---- -layout: post -title: Looking forward to Codestock 2017 -date: 2016-11-09 07:53:36.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- developer -- event coverage -tags: [] -meta: - _oembed_97545764ae6ff19ad65ec209cd303c61: '
Front-page' - _oembed_time_97545764ae6ff19ad65ec209cd303c61: '1478695179' - _edit_last: '1' - _oembed_4792ffcf47cb5692b89968b58dc086dc: - _oembed_time_4792ffcf47cb5692b89968b58dc086dc: '1478695563' - _thumbnail_id: '466' - _oembed_f8afe1d83be1a234b2acd62eea55929c: - _oembed_time_f8afe1d83be1a234b2acd62eea55929c: '1484577621' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2016/11/looking-forward-to-codestock-2017/" ---- -
As we were preparing for a webinar yesterday, a co-worker, @nicksanjines, mentioned a TN local developer event: Codestock
-Side note: Nick has a great set of examples on Jenkins, Android Studio, and Calabash at thedieseldeveloper.com
-Not only have I been looking for an excuse to get down there in 2017, this looks like a very legit event. Consider Jared Smith, organizer of HackUTK and security researcher. That's one of dozens of speakers at an event that attracted 900 devs.
-Just like Abstractions.io this past year, I am ready for my mind to be blown.
-https://www.youtube.com/watch?v=AzjRmqR1wTU
diff --git a/_posts/2016-11-12-how-can-a-pid-controller-help-to-fight-eula-apathy.html b/_posts/2016-11-12-how-can-a-pid-controller-help-to-fight-eula-apathy.html deleted file mode 100644 index 7331c32..0000000 --- a/_posts/2016-11-12-how-can-a-pid-controller-help-to-fight-eula-apathy.html +++ /dev/null @@ -1,93 +0,0 @@ ---- -layout: post -title: How can a PID Controller Help to Fight EULA Apathy? -date: 2016-11-12 20:57:55.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- digital literacy -- EULA -- privacy -- social -tags: [] -meta: - _edit_last: '1' - _oembed_33fb5e1868eddc701fe91354ff37057c: "{{unknown}}" - _oembed_f74905627c1f89a4c125d16b833730dd: - _oembed_time_f74905627c1f89a4c125d16b833730dd: '1478974965' - _oembed_440d16d673d9e306e1bed9fa338ddd05: - _oembed_time_440d16d673d9e306e1bed9fa338ddd05: '1478975062' - _oembed_7b081a07b2b4dc59e2f5fa8f5eace19a:- _oembed_time_7b081a07b2b4dc59e2f5fa8f5eace19a: '1478979789' - _thumbnail_id: '477' - _oembed_224b1c13d437ce34de70417f61804a19: - _oembed_time_224b1c13d437ce34de70417f61804a19: '1484577621' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2016/11/how-can-a-pid-controller-help-to-fight-eula-apathy/" ---- -Digital literacy, something we need! @paulsbruce - @kinlane @apistrat - #meetup @hillholliday - #APIStrat #API - #Digital #Digitalliteracy - pic.twitter.com/mYLjU9P38Y
— MaureenMansfieldALM - (@MaureenManALM) November - 4, 2016
Value comes from what you do. As part of a working class, I firmly believe that when someone takes away your control over the work you do, they violate a basic human right. Work hard, work smart, but work on something that people need. That is value.
-I'm seriously disturbed by wide-spread apathy over the legal ramifications of EULAs on personal rights. This is core curriculum IMO for digital literacy. If you own a phone or use email, you have very few digital rights, and that's a problem. We've been eating marshmallows, lots and lots of marshmallows.
-What is a EULA?
-An End User License Agreement (EULA) is that it's that thing you press the "Agree", "Accept", or "I Understand" even when you didn't read the fine print. Think of every app on your phone, every e-commerce store, and every piece of software installed on your computer. It is a legal document, binding and enforceable and you have no idea what it says.
-What is EULA apathy?
-Well, consider that you have accepted many of them, probably without knowing it, and the terms can be changed at any time without requiring your consent. Meh. That's EULA apathy.
-If this doesn't bother you, think of all the photos and videos you, your friends, and family upload to social networks daily. You don't legally own them. That means that you can find yourself, your children, and intimate details of your house on display in a designer picture frame at your nearest Target store.
-If today, your mobile phone or cable provider think you're using their service in a way that doesn't fit their current perspective, they cut you off. No internet, no phone, no ability to communicate to the outside world. What is free speech without the ability to speak? This increasingly matters, not just in China, but everywhere.
-What is the *human* problem with EULAs?
-The problem is delayed gratification, particularly with small-transaction agreements such as mobile apps. It's that new game, the app that helps you find food or gas or sex, connect with others, something that makes them feel just a bit more validated than they are now. Hello, all of social media.
-[caption id="attachment_480" align="alignright" width="226"]
Why does National Geographic need my phone number, my location, and my files?[/caption]
We accept what we don't understand because our immediate gain outweighs future loss. EULAs are the digital equivalent of sugar addiction in America; our tipping points in both fights has long since passed. And just like our biological imperative to optimize for caloric intake, our emotional imperative for validation by other humans drives us to ignore what businesses ask from us.
-How the fuck do you compete with human nature? How do you get people to care about what they don't understand?
-After a long time, I realized...you don't.
-My problem with EULAs is really about corporate personhood.
-EULAs are designed to protect the rights of the business over the individual. They often ensure that you are responsible for bad things, illegal materials, and inappropriate use of a thing or service, but importantly that you do not own the content you provide. Not coincidentally, they also absolve the licensor (the business) of any wrong-doing to minimize risk to profit.
-This leads to very unethical behavior, such as Facebook ignoring that many users are selling deadly weapons on their service. If the business owns the data but the creator is still held responsible for how it is used, we write an ethical blank check to corporations that we know will choose profit over well-being...every time.
-Ownership infers responsibility. EULAs are conscription law. Digital fucking slavery.
-If EULAs are a problem, how do we fix them?
-The reality is that we can't start by trying to change businesses or legal structures easily. If you can provide an easier way, people will be far more likely to take it. EULAs are about risk mitigation, an outcome of greed. You just have to get human nature to overcome the problem WITH ITSELF.
-We can fight greed with greed by letting people control profits from their own data.
-I brought this up a little last week as host of our Boston APIcraft meetup. I asked the question "If we quantified profitability over our personal data, would people be more inclined to pay attention to their rights (i.e. personal value)?" The conversation was positive, and the group was definitely there when it came to digital literacy. But the first step is personal data ownership for sure.
-
Ownership is fundamental to profitability when it comes to your own data. Let's assume we get over how the hell to individually own and host our own data stores in a way that vendors even want to be involved in. Let's skip past that part, thought Phil Windley has some interesting thoughts into this (i.e. PICO and personal clouds).
-Once we own our own data, we have control and responsibility over it. Then we can make some real money progress.
OKAY, OKAY! How does PID control logic apply to EULAs?
-Watching a GOTO; presentation on agile by Dave Thomas last night, he discussed how much of our modern world consists of PID controller circuits. He used an example of how mega-boats solved their human navigation problems before GPS was a thing.
-The reason is that in many situations, making a decision without at least a bit of historical context or tracking to future goal often leads to failure. Sounds exactly like what is missing from our current situation with EULA apathy.
-https://www.youtube.com/watch?v=0vqWyramGy8
-Let's apply the PID pattern to personal profitability. Consider the three active elements in the PID control model:
-Proportion, the current state of the system:
-Your location and preferences are being traded privately between commercial entities. Since you accepted the EULAs that consign data ownership to them, they can do whatever the fuck they want with it. Fitbit and the insurance industry comes to mind. They make money, you see none of it. You pay for the device and see only a fraction of the value.
-We want to change this, but pause, this is simply what is happening now, current state.
-Integral, the history of profit:
-How much money have people been making on your data, and have we been able to change the tide in other circumstances? Lots, and yes we have.
-Global BI and analytics market is estimated at $16.9 billion, says Gartner...you know, if you believe anything they say. Sometimes I do. On Thursdays.
-"The global mobile analytics market is expected to grow from USD 1.36 Billion in 2015 to USD 4.12 Billion by 2020, at a Compound Annual Growth Rate (CAGR) of 24.73% during the forecast period." [ref] - that's just mobile though.
-Translation: pants on fire, slap me rotten, nucking futs profits.
-And that's not even the money you spend on stuff, that's money made on the data about the stuff you spend your money on, and what you don't. The reason that Google search, Facebook, and news sites are free is because in exchange for some a little service, they sell a massive amount of rich data about you.
-We have changed this in the past (patient rights), but we have done this through legal channels which in America is a veil of ignorance and infects every good idea it touches with bureaucratic impotency.
-We also have changed the way enterprise software profits. The web runs on approximately 75% linux; open source software disrupts the profits of big bads.
-Derivative, the future goal:
-If we want people to profit off their own data, we have to find alternatives to the broken systems we have now. It needs to be cheap. It needs to be transparent. And it needs to be easy to manage and integrate with. Git comes to mind.
-It's far more than a single technology though, it's an ecosystem, and it needs compelling arguments for the average citizen to move forward. If only we had a government that didn't let institutions off easily for massive data breaches, maybe businesses wouldn't want to take on the liability of storing all our data in the first place.
-We also need to deal with the problem of access to this data. If net neutrality is really dead, then there's no other solution but to maintain our data on someone else's infrastructure (not our own), but due so in a yet-undeveloped fully encrypted manner.
-It's not as simple as a PID controller, but no human problem ever is. The best we can do is engage, try to move some part of it forward, and that's EULAs for me. As always, more to come.
diff --git a/_posts/2016-11-16-schrodingers-box.html b/_posts/2016-11-16-schrodingers-box.html deleted file mode 100644 index ae9b08e..0000000 --- a/_posts/2016-11-16-schrodingers-box.html +++ /dev/null @@ -1,39 +0,0 @@ ---- -layout: post -title: Schrödinger's Box and other complexities of scaling the software delivery lifecycle -date: 2016-11-16 18:58:47.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- DevOps -- SDLC -tags: [] -meta: - _edit_last: '1' - _oembed_70c57d2c5345d489627ea7baaf6dbf0e: "{{unknown}}" - dsq_thread_id: '' - _wp_old_slug: schrodingers-box-and-other-complexities-of-scaling-the-software-delivery-lifecycle - _oembed_20296ea35c89b5180549f1f06d16df25: - _thumbnail_id: '496' - _oembed_time_20296ea35c89b5180549f1f06d16df25: '1479738088' - _oembed_63c55cf3cd5cd706aa91e36f4b82b28f: - _oembed_time_63c55cf3cd5cd706aa91e36f4b82b28f: '1484577621' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2016/11/schrodingers-box/" ---- - -Thank you to everyone at Defrag for attending my session. Great conversations afterwards, and looking forward to any more input you have in the future!
-[embed]https://www.youtube.com/watch?v=K-ZCQVuys1s[/embed]
--
Tweets about @paulsbruce at DefragX 2016
-
For my presentation at AnDevCon SF 2016, I focused on how Espresso represents a fundamental change in how approach the process of shipping software that provably works on a mobile ecosystem that is constantly changing.
-The feedback was overwhelmingly good, many people who stopped by the Perfecto booth before or after our talk came to me to discuss topics I raised. In other words, it did what I wanted, which was to provide value and strike up conversation about how to improve the Android UI testing process.
-If you're pressed for time, my slides can be found below or at:
-bit.ly/espresso-edges-andevcon
https://youtu.be/h85ZaRvG2mU
- -diff --git a/_posts/2017-01-06-installing-android-sdk-in-docker.html b/_posts/2017-01-06-installing-android-sdk-in-docker.html deleted file mode 100644 index 33bdd80..0000000 --- a/_posts/2017-01-06-installing-android-sdk-in-docker.html +++ /dev/null @@ -1,66 +0,0 @@ ---- -layout: post -title: Installing Android SDK in Docker -date: 2017-01-06 13:35:18.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- Docker -- Jenkins -tags: -- Android -meta: - _edit_last: '1' - _thumbnail_id: '516' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2017/01/installing-android-sdk-in-docker/" ---- -
For a recent project, I had to include the Android SDK build tools as part of a Jenkins Dockerfile. No problem. Download and execute installer, right?
-Wrong. With Google installers come license agreements, and I needed a way to reliably accept the terms and conditions of the installer and it's dependencies automatically. Here's what my Dockerfile looks like:
-FROM jenkinsci/jenkins:2.0-beta-1 -USER root -RUN mkdir /var/log/jenkins -RUN mkdir /var/cache/jenkins -RUN chown -R jenkins:jenkins /var/log/jenkins -RUN chown -R jenkins:jenkins /var/cache/jenkins - -RUN apt-get update && apt-get install -y apt-transport-https -RUN apt-get -q -y install lsof - -RUN wget https://dl.google.com/android/android-sdk_r24.4.1-linux.tgz -O /opt/android-sdk.tgz -RUN tar zxvf /opt/android-sdk.tgz -C /opt/ -RUN rm /opt/android-sdk.tgz - -RUN >/etc/profile.d/android.sh -RUN sed -i '$ a\export ANDROID_HOME="/opt/android-sdk-linux"' /etc/profile.d/android.sh -RUN sed -i '$ a\export PATH="$ANDROID_HOME/tools:$ANDROID_HOME/platform-tools:$PATH"' /etc/profile.d/android.sh -RUN . /etc/profile -RUN apt-get install git-core -RUN ( sleep 5 && while [ 1 ]; do sleep 1; echo y; done ) | /opt/android-sdk-linux/tools/android update sdk --no-ui --filter platform-tools,android-24,build-tools-24.0.1,tools,extra-android-support,extra-android-m2repository -RUN chmod -R 755 /opt/android-sdk-linux -RUN dpkg --add-architecture i386 -USER jenkins -ENV JAVA_OPTS="-Xmx8192m"-
Not complicated, really, but worth documenting for others out there.
-Backwards-compatibility for Build Tools in Jenkins
-I have 'build-tools-24.0.1' in there because the app I'm working with has not been upgraded to the latest version of Gradle, but it's worth noting too because not everyone has the luxury of changing code/compile settings just because Google ships new binaries. Thanks Google, you really know how to break my build. ;*
-Instead, I chose to own the responsibility of the version of tools needed on my build nodes for the types of projects I intend to compile on them. Because of this, I need to know the specific android update sdk filter codes that correspond to the pretty package names I see on my workstation in SDK Manager.
-To list the codes that you might need in your own update filter, use the following command under your Android SDK tools folder:
-android list sdk -a -e-
...which displays a list like this:
-
That is all.
-Semi-related Reading:
--
-
- Espresso is Awesome, but Where Are It's Edges? -
- Android Studio: How to find the Package Name from APK -
- Why Espresso: Unit vs. UI testing -
For an reference example, I had to set up Jenkins to build my Android app. Though I'm using a Mac, once Docker is involved, I can also use the exact same steps on my Windows machine too, and so can you.
--
-
- Install Docker -
- Use my existing Docker setup files
-
-
-
- Grab the contents of this GitHub folder to your machine -
- Change directory to where you saved the above contents -
- Build the docker files to create images. Run:
-
docker build -t jenkins-data -f Dockerfile-data . -docker build -t jenkins2 .
-
- - Run these freshly minted Docker images as a new container:
-
-
docker run --name=jenkins-data jenkins-data -docker run -p 8080:8080 -p 50000:50000 --name=jenkins-master --volumes-from=jenkins-data -d jenkins2
-
-
- - In a browser, log in to your Jenkins instance at http://127.0.0.1:8080 -
- Complete the initial Jenkins setup by walking through the on-screen prompts -
Much thanks to Sha who wrote this article that quickly highlights the steps for getting Jenkins 2.0 running on Docker. All I added to my Dockerfile was steps to install the Android SDK so that Jenkins can build my app.
-Related reading:
- -diff --git a/_posts/2017-01-13-locale-bugs-currency-formatting-in-android-studio.html b/_posts/2017-01-13-locale-bugs-currency-formatting-in-android-studio.html deleted file mode 100644 index eff4e27..0000000 --- a/_posts/2017-01-13-locale-bugs-currency-formatting-in-android-studio.html +++ /dev/null @@ -1,48 +0,0 @@ ---- -layout: post -title: Locale bugs & currency formatting in Android Studio -date: 2017-01-13 10:00:18.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- Android -- Espresso -- pitfalls -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '545' - dsq_thread_id: '' - _wp_old_slug: locale-bugs-currency-formatting-and-other-android-pitfalls -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2017/01/locale-bugs-currency-formatting-in-android-studio/" ---- -
I grew up in the United States and like most countries I've traveled to, I'm used to seeing money that look like this: $56.33 €12,50 £281.71
-A modern IDE tells you when you're ignorant
-Today, Android Studio was kind enough to tell me that some sample code had a problem: "Implicitly using the default locale is a common source of bugs". Nice.
-
Looking at this sample code, I asked myself: "Why should the UI format this number, isn't that going to hide what the underlying application logic is saying?"
-So it was a great thing that Android Studio shows little messages like this because it got me thinking about how to properly handle currency in the view layer.
-Simply changing your app code won't save you!
-This is a case where view layer formatting isn't the appropriate place to deal with rounding. The business logic of this application, the algorithm that calculates tips, should ultimately be in charge of rounding to the locale-appropriate currency decimal place, in Java this means using the BigDecimal class for manipulating values. So, I forked my own copy and updated the sample to remove the view formatting code.
-Then I changed the locale on my device, and my Espresso tests started breaking.
-See, when you write your format codes and test logic under a narrow mindset of one locale/language/currency (en_US), the test data you use can break your tests if the app is run under a different locale (ar-IQ) which format things like currencies (USD, IQD) to a different arithmetic precision (the dollar uses 2 decimal places, the Dinar uses 3). [locale/currency lookup table]
-An example of an Espresso test written assuming western currencies can be found below. Dinar has three decimal places, so this particular sample fails because values are handled as doubles (floating point numbers). Without controller logic to deal with rounding, it comes out as 35.912 which is not the same as the 2-decimal data "35.91" in my test code.
To simplify things, much of the code is written using doubles to pass currency around, even though this isn't best practice. Even still, so long as we use BigDecimal to handle the higher-order calculations, we can downgrade the decimal precision in outbound double values to the view layer. Then we have the option of using locale-accurate test data, managing the precision in our tests as well.
-Check it out for yourself...
-If you want to spin these examples up yourself, you can clone my repo. Also, if you want to see this work in continuous integration, check out my article on running a Jenkins / Android build server on Docker.
-Reference:
--
-
- Locale and Currency Reference List -
- How to resolve a Java rounding double issue -
- Why not use double or float to represent currency? -
- Which rounding mode to be used for currency manipulation in java? -
My brave, introverted wife marched yesterday in Boston while I took the kids to the park and cleaned up the house. It's the least we both could do.
-
After a political turnover that can only be compared to falling down a rabbit hole and ending up back in the 50s in a communist Russia run by a crotchety, ignorant oompa loopa, this is what America has come to. Some of our most intelligent, kind, and confident people having to protest for freedom, upholding basic human rights, and against bigotry.
-America: So Bad, Even Introverts Rally
-I am my wife's husband. We are very different. The way she recharges is to stay in doors and knit something. I like to stay out late and meet new people. But that's what she did yesterday. She went with an extroverted friend and surrounded herself with 110,000 other people because involvement matters. She hugged a college student and they both started crying together in public.
-
That's why she's brave in my book, because in her book these kinds of extroverted things don't happen, but they do when it matters.
-Since I wasn't there, I didn't get to see city residents supporting the cause from their windows stories above the march. All my worst fears after the Boston marathon bombing were unnecessary because there was not a single reported negative incident. Service vehicles that blocked off the streets from traffic honked their horns in solidarity as people proudly walked by...that's what my wife came back with, a story of hope and meaning.
-After over a year of thinking about what lead up to this, I would like to offer a few thoughts, and one big suggestion.
-Leaders who actually lead deserve our attention
-Most of our current representative government is awful. They're just awful. Both sides, but what I really mean by 'most sides' is most of the majority side.
-What other options to we have? Let's recap.
-Elisabeth Warren takes to task any politician who practices corruption, champions education and equality, or supports the subjugation of others. Why the fuck isn't she our president already? The short answer is that for all her rights, America is still so sexist (even women) and she's so left that it would be hard to expect a majority of voters not to mention our current political complex to accept her...
-...but oh wait, Washington just elected the Actual Antichrist (if you believe in that sort of hoo haw), so I guess it's entirely possible to get a left-wing presidential candidate who is overqualified and underrepresented elected in 2020. Let's focus our party efforts to do just that.
-[embed]https://www.youtube.com/watch?v=0oGzqrVlxf0[/embed]
-Or how about Bernie Sanders, the candidate that we the people never got the chance to vote for. During protests across the world yesterday against This Desolation Presidency, our Bern spoke out for human rights in his home state.
-Are you kidding me? I actually feel a little mentally disabled when I think about how this guy lost in a race against Fat-Fingers McGrabspussy. Are we blind, deaf, dumb, stupid, or all of the above for not giving him an oval room to run the free world?
-If a majority of white America can't yet accept that a women should truly be leading us as a global power at this point, then Sanders was the most obvious and amenable solution to our current State of Bias.
-https://www.youtube.com/watch?v=KViUIe4jPR8
-Accept and move beyond our mistakes
-We can't ignore our mistakes without being doomed to make them again.
-The Democratic party gambled on a risky political figurehead and lost BIG. There was enough time and examples of Russian-colluding rhetoric before the primaries to expect that the presidential race would be fought dirty, and picking a candidate carrying so many possible vectors of attack was a poor choice no matter how progressive or technically qualified they were.
-DNC leadership failed not only its constituents, but all of America by driving the Republican party to put up such poor choice traditional candidates such as Ted Cruz and Ben Carson that conservative voters would rather side with a small-minded (and small-handed) demagogue than ever consider voting for the other side.
-We have to fix that now, there is no good 2020 outcome without a reboot.
-Fight propaganda with informational literacy
-Disinformation also needs to be a solved problem by next time around. I'm not talking about simply plugging our security gaps, I'm talking about educating people on how to filter opinion from fact. The battle is fought one person (not article) at a time.
-Informational literacy enables people to form their own ability to think rationally and critically about what's being presented to them. Everyone is biased, and everyone deserves to work their own flaws out; but being able to agree upon basic facts starts by teaching people (especially those still in formative stages) how to distinguish and agree upon facts.
-Listening is also a mandate next time around. Confusion and intolerance about a bad idea is fine, but let that go to far and it blinds you to what may actually be going on. Free speech applies to idiots, villains, thieves, and zealots too, sadly. Clearly, I'm okay with name-calling and sometimes irrational arguments. but keep in mind that shutting people out of communicating their bad ideas doesn't kill the bad idea; it just incubates it.
-A Solution for Next Time Around: Inclusion Beyond Under-representation
-So let's do this. Let's adjust the strategy to include, not exclude, both our hopes and our reality. Let's continue to fight for human rights, stand up for others who don't have the same level of privilege as us, and remember that even people with bad ideas are people who vote.
-We can and will hijack enough mindshare in the next four years to vote based on better ideas, but it will take patience, tolerance, and perseverance. Who knows, maybe we'll learn something about how to change people instead of just saying that we can. That's a party I'll belong to.
-More Reading:
--
-
- 7 Practical Tips for Inclusion -
- When you are really invested, you worry -
- HackCU: An Example of Student Leadership -
I've heard that React Native is really cool. I've heard it can help to change your delivery, team, and hiring strategy. I've also heard it's toolchain is immature and that it's not 'enterprise friendly'.
-Extraordinary claims require extraordinary proof, so I decided to put it through the paces I see enterprises require every day. Namely, how do you use a cloud lab in situations where you're debugging a critical incident found in production?
-Since I have a variety of real mobile devices at my disposal in the Perfecto cloud, let's see how quick it is to connect React Native to one of them!
-Side question: what was the last physical mobile device you needed to debug an issue on a specific platform, carrier, or form factor?
-Click here to tweet me your answer!
Running React Native Code on a Real Device
-Sitting in the back row of a local meetup, I quickly installed the requisites on my MacBook, launched a Perfecto device, and was up and running. Like all bootstrap activities, this was flawless. Then I ran the usual 'react-native run-android':
-
First snag, unlike Android Studio, the magic dust that ships with React to automate the Gradle build and deploy process was lacking the -s command, which of course failed the build process. The maturity of React tooling is a side-topic, but all we need is to amend that parameter with a device serial number.
-
Listing the devices, we see that my cloud device correctly registers in ADB:
-
After copying the ADB command and rerunning with the -s argument added, the app ran, but with some debugging connectivity issues.
Debugging React Native on a Different Network
-
What this message is telling us is that stunnel is configured to allow our computer to see the device, but not the other way around.
-Since React Native debugging needs to load javascript hosted by your workstation, we'll need to point debugging on the device to an address that resolves to your workstation's address. I use ngrok for this.
-./ngrok http 8081
-This produces a dynamic hostname that will forward all incoming traffic on port 8081 down to my local workstation where the React server is running.
-To get to the React Native developer tools on the device to have the right debugging server address, I simulate a device shake by sending a keypress 82 command, then navigate to 'Dev settings' and 'Debug server host & port...':
-
-
And voila! A React Native app on a real Samsung Galaxy S7 device hosted in the Perfecto cloud running in debug mode!
-
Could this be easier? Sure, if React Debugging used ADB to do all debugging interactions, but that would mean a lot of re-architecting on their part. The simple fact is, React Native debugging dictates access to the developer workstation via HTTP, so ngrok is kind of necessary due to their tooling.
-Next steps:
-You could automate the IP configuration in AppDelegate.m like this walkthrough does if you want to.
-You could probably even grep the dynamic host name in a shell command and write it dynamically to that file before React Native deployment to the device. But that would be another article.
-More reading:
- -diff --git a/_posts/2017-02-13-fast-feedback-at-developer-week-2017.html b/_posts/2017-02-13-fast-feedback-at-developer-week-2017.html deleted file mode 100644 index a52582c..0000000 --- a/_posts/2017-02-13-fast-feedback-at-developer-week-2017.html +++ /dev/null @@ -1,34 +0,0 @@ ---- -layout: post -title: Fast Feedback at Developer Week 2017 -date: 2017-02-13 20:49:12.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- automation -- developer -- DevOps -- SDLC -tags: [] -meta: - _oembed_time_420abdc315bb0a46391e27482d9f4dbf: '1487351857' - _oembed_420abdc315bb0a46391e27482d9f4dbf: - _oembed_375d16120d7d307cde8104bd49efad4a: "{{unknown}}" - _edit_last: '1' - _thumbnail_id: '648' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2017/02/fast-feedback-at-developer-week-2017/" ---- -
I spoke about the importance of fast feedback across the software delivery pipeline at Developer Week 2017 on Tuesday. Below is the first 30m of the talk plus my slides which include links to my research.
-https://youtu.be/nkg5dcL57Wg
-My slides are available here: bit.ly/2klfZvu
- diff --git a/_posts/2017-04-19-android-debug-features-and-tools-chat-with-sam-edwards.html b/_posts/2017-04-19-android-debug-features-and-tools-chat-with-sam-edwards.html deleted file mode 100644 index 9b85df9..0000000 --- a/_posts/2017-04-19-android-debug-features-and-tools-chat-with-sam-edwards.html +++ /dev/null @@ -1,46 +0,0 @@ ---- -layout: post -title: Android Debug Features and Tools chat with Sam Edwards -date: 2017-04-19 09:33:08.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- Android -tags: -- informal interface -meta: - _edit_last: '1' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2017/04/android-debug-features-and-tools-chat-with-sam-edwards/" ---- -Developing Android apps can be hard...without the right tools and patterns in your back pocket.
-I had a chance to sit with Sam Edwards (@handstandsam) to talk about some of the work that went into his presentation at Droidcon Boston last week. There are a bunch of tools outside in out-of-box Android stack that I wasn't aware of (see below), but Sam quickly educated us. His full talk will be available in a few weeks once the conference dust settles.
- -Improving the Espresso Testing Landscape
-Afterwards, Sam and I went for a stroll down Tremont and discussed some patterns people were applying to simplify writing Espresso tests.
-Shauvik Roy Choudhary who I met at last year's Capital One Android Summit has produced some great work around Testing Robots (slides and code). This is not to be confused with Jake Wharton's work on (also named) Testing Robots, equally awesome and worth your time reviewing if you haven't already. Shauvik also produced Barista, which you can find in the app store.
-Related note: in recent StackOverflow work, I found out about another cool project called Barista, a library of wicked helpful functions for Espresso testing by Rafa Vázquez, Roc Boronat, and Sergi Martínez.
-Great to see people sharing their experiences and lessons learned so we don't have to toil on them ourselves.
--- -Slides are now up from my talk at #droidconbos - "It's an Inside Job" - Building Debug Features: https://t.co/ovBdFnOa8e … #androiddev pic.twitter.com/Bp0ipiIdc1
-— Sam Edwards (@HandstandSam) April 12, 2017
Android debug and design tools that Sam covered:
- -diff --git a/_posts/2017-04-25-dont-panic-or-how-to-prepare-for-iot-with-a-mature-testing-strategy.html b/_posts/2017-04-25-dont-panic-or-how-to-prepare-for-iot-with-a-mature-testing-strategy.html deleted file mode 100644 index 91f17f5..0000000 --- a/_posts/2017-04-25-dont-panic-or-how-to-prepare-for-iot-with-a-mature-testing-strategy.html +++ /dev/null @@ -1,36 +0,0 @@ ---- -layout: post -title: Don't Panic! (or how to prepare for IoT with a mature testing strategy) -date: 2017-04-25 10:33:44.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- DevOps -- IoT -- testing -tags: [] -meta: - _edit_last: '1' - dsq_thread_id: '' - _thumbnail_id: '678' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2017/04/dont-panic-or-how-to-prepare-for-iot-with-a-mature-testing-strategy/" ---- -
Thanks everyone for coming to my talk today! Slides and more links are below.
-As all my presentations are, this is meant to extend a dialog to you, so please tweet to me any thoughts and questions you have to become part of the conversation. Looking forward to hearing from you!
-More links are in my slides, and the presenter notes are the narrative in case you had to miss part of the presentation.
- --
-
- Factors: Digital Coverage Index and More by Perfecto 2017 Q2 -
- A Developer's Primer to Build-time Quality -
- [pre-release] The Path to Releasing Confidently in DevOps -
- Signup for my DevOps and Continuous Delivery Insights newsletter -
I recently had the opportunity at a conference to ramen up with Lance Gleason of Polyglot. Of the many things we discussed in 3hrs, it came down to this:
-"A really good developer implicitly cares about the quality of their work."
-Forget what you've previously read, the blah blah, the pious statements, the bullshit for a moment. Consider these four reasons, practices if you will, that explain the statement above.
-1. Write code that makes sense next week
-I can't reason about things that don't fit in my head. So please write code that fits in my head, even if it already fits in yours. I say that understanding how big my own head is sometimes, and maybe that applies to you too. When we do this, code is easy to reason AND collaborate about, and it turns out that's severely important to our craft.
-Also keep in mind that the amount of work you have to do next week will make you forget what you did this week. I love to forget about things, it helps me prioritize life. I wish the same for you. All code is an expression of intent, and when it contains bugs, we have to fix them. That means what we do today has to be simple enough to waste no time fixing when we revisit it in the future.
-2. Write code that contributes to a purposeful existence
-Life quickly becomes meaningless when you're working on meaningless stuff. Figure out what matters most in your life, make that your goal, and write code that contributes to that goal. Don't become a programmer for the money because you'll quickly learn to hate the programmers around you because we can be prickly unless you understand why we are what we are.
-Personal scrutiny is a blessing and a curse. Self doubt, imposter syndrome, and uncertainty are all signs that you still have a soul, so don't worry if you see them while you write code. So when you see them on your journey with code or when someone points something out, you have no obligation to accept them verbatim; but you should receive it just as you should receive compliments, consider it, then if it makes sense work it into your code.
-Be ethical enough to say "I will not build this" every time your conscience tells you that your product isn't going in a direction. Every time. Why? Because there is no soul in software other than what we as humans imbue into the process. Businesses are paper. You are craftsman. Be that.
-3. Write code that entices others to collaborate
-Who likes to toil away in obscurity on something no one else cares about? Exactly, so find projects that others also care deeply about, enough to contribute to, then go to it hard. Said the opposite way, you know if you're working on something valuable when others are so interested in it that they also want to invest their time in it.
-Don't write negative comments over someone else's work. Just don't. Establish a dialog that helps you understand why they did what they did. Then if you still don't agree, be clear about the obstacles you see in the current approach and find which ones are imperative to resolve. Be okay with things that aren't perfect so long as there's a way to revisit them later.
-You know you're doing it right when you leave no code behind that others are afraid to question. This is a characteristic of mature engineering, the self-evident latitude in your work for future ideas to improve what you have crafted.
-4. Write code that anticipates change
-Everything changes. Flexible code enables us to correct mistakes when we realize them in the future. Smaller code is easier to understand and fix.
-Be clear about your surface area. The more you know about the limits of code, the easier it is to build code that uses it properly. If it's a function or method, naming and sufficient typing clarifies it's purpose and limits. If it's a REST API, a machine-readable descriptor with examples helps people integrate this into other code more accurately.
-Constraints and dependencies also communicate the flexibility of a system. If you know that the code your writing shouldn't be used in specific circumstances, document those in a way that is easier for others to consume. Isolate dependencies when they are insufficiently abstract such that you can't write code that 'fits in your head'. Vet external software that you don't own, yes, open source libraries to ensure they meet licensing, security, and performance requirements.
-Don't just talk about quality, practice it.
-Quality isn't something you 'build in' afterwards, it's desirable characteristics exhibited by your work. There are many ways to accomplish this, but all take practice and focus, trial and error. Exercise teaches us to reason about why we do what we do, not just what we're doing. True software craftsmanship produces sensible, purposeful, collaborative, flexible code.
diff --git a/_posts/2017-06-27-automating-the-quality-of-your-digital-front-door.html b/_posts/2017-06-27-automating-the-quality-of-your-digital-front-door.html deleted file mode 100644 index 47b03ed..0000000 --- a/_posts/2017-06-27-automating-the-quality-of-your-digital-front-door.html +++ /dev/null @@ -1,172 +0,0 @@ ---- -layout: post -title: Automating the Quality of Your Digital Front Door -date: 2017-06-27 12:05:50.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- automation -- DevOps -- IoT -- Jenkins -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '693' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2017/06/automating-the-quality-of-your-digital-front-door/" ---- -Mobile is the front door to your business for most / all of your users. But how often do you use your front door, a few times a day? How often do your users use your app? How often would you like them to? It’s really a high-traffic front door between people and you.
-This is how you welcome people into what you’re doing. If it’s broken, people don’t feel welcome.
-[7/27/2017: For my presentation at Mobile Tea Boston, my slides and code samples are below]
-- -
Slides with notes: http://bit.ly/2tgGiGr
-Git example: https://github.com/paulsbruce/FingerprintDemo
The Dangers of Changing Your Digital Front Door
-In his book “On Intelligence”, Hawkins describes how quickly our human brains pick up on minute changes with the analogy of someone replacing the handle on your front door with a knob while you’re out. When you get back, things will seem very weird. You feel disoriented, alienated. Not emotions we want to invoke in our users.
-Now consider what it’s like for your users to have you changing things on their high-traffic door to you. Change is good, but only good changes. And when changes introduce problems, forget sympathy, forget forgiveness, people revolt.
-What Could Possibly Go Wrong?
-A lot. Even for teams that are great at what they do, delivering a mobile app is fraught with challenges that lead to:
--
-
- Lack of strategy around branching, merging, and pushing to production -
- Lack of understanding about dependencies, impacts of changes -
- Lack of automated testing, integration woes, no performance/scalability baselines, security holes -
- Lack of communication between teams (Front-end, API, business) -
- Lack of planning at the business level (marketing blasts, promotions, advertising) -
Users don’t care about our excuses. A survey by Perfecto found that more than 44% of defects in mobile apps are found by users. User frustrations aren’t just about what you designed, they are about how they behave in the real world too. Apps that are too slow will be treated as broken apps and uninstalled just the same.
-What do we do about it?
-We test, but testing is a practice unto itself. There are many test types and methodologies like TDD, ATDD, and BDD that drive us to test. Not everyone is cut out to be a great tester, especially when developers are driven to write only things that works, and not test for when it shouldn't (i.e. lack of negative testing).
-[caption id="attachment_701" align="alignright" width="279"]
Allistar Scott - Test 'Ice Cream Cone'[/caption]
In many cases, automation gaps and issues make it easier for development teams to fall back to manual testing. This is what Allistar Scott (of Ruby Waitr) calls the anti-pattern 'ice cream cone', an inversion of the ideal test pyramid, and Mike Cohen has good thoughts on this paradigm too.
-To avoid this downward spiral, we need to prioritize automation AND which tests we chose to automate. Testing along architecturally significant boundaries, as Kevin Henney puts it, is good; but in a world full of both software and hardware, we need to broaden that idea to 'technologically significant boundaries'. The camera, GPS, biometric, and other peripheral interfaces on your phone are a significant boundary...fault lines of the user experience.
-Many development teams have learned the hard way that not including real devices in automated testing leaves these UX fault lines at risk of escaping defects. People in the real world use real devices on real networks under real usage conditions, and our testing strategy should reflect this reality too.
-The whole point of all this testing is to maintain confidence in our release readiness. We want to be in an 'always green' state, and there's no way to do this without automated, continuous testing.
-Your Code Delivery Pipeline to the Rescue!
-Confidence comes in two flavors: quality and agility. Specifically, does the code we write do what we intend, and can we iterate and measure quickly?
-Each team comes with their own definition of done, their own acceptable levels of coverage, and their own level of confidence over the what it takes to ship, but answering both of these questions definitively requires adequate testing and a reliable pipeline for our code.
-
Therein lies the dynamic tension between agility (nimbleness) and the messy world of reality. What’s the point of pushing out something that doesn’t match the needs of reality? So we try to pull reality in little bits at a time, but reality can be slow. Executing UI tests takes time. So we need to code and test in parallel, automate as much as possible, and be aware of the impact of changes on release confidence.
-The way we manage this tension is to push smaller batches more frequently through the pipeline, bring the pain forward, in other words continuous delivery and deployment. Far away from monolithically, we shrink the whole process to an individual contributor level. Always green at the developer level...merge only code that has been tested automatically, thoroughly.
-Even in a Perfect World, Your Front Door Still Jams
-So automation is crucial to this whole thing working. But what happens when we can’t automate something? This is often why the “ice cream cone” exists.
-Let’s walk through it together. Google I/O or WWDC drops new hardware or platform capabilities on us. There’s a rush to integrate, but a delay in tooling and support gums up development all the way through production troubleshooting. We mock what we have to, but fall back to manual testing.
-This not only takes our time, it robs us of velocity and any chance to reach that “always green” aspiration.
-The worst part is that we don’t even have to introduce new functionality to fall prey to this problem. Appium was stuck behind lack of iOS 10 support for months, which means most companies had no automated way to validate on a platform that was out already.
-And if anything, history teaches us that technology advances whether the last thing is well-enough baked or not. We are still dealing with camera (i.e. driver stack) flakiness! Fingerprint isn’t as unreliable, but still part of the UI/UX. And many of us now face an IoT landscape with very few standards that developers follow.
-So when faced with architectural boundaries that have unpolished surfaces, what do we do? Mocks...good enough for early integration, but who will stand up and say testing against mocks is good enough to go to production?
-IoT Testing Provides Clues to How We Can Proceed
-In many cases, introducing IoT devices into the user experience means adding architecturally significant boundaries. Standards like BLE, MQTT, CoAP and HTTP provide flexibility to virtualize much of the interactions across these boundaries.
-In the case of Continuous Glucose Monitoring (CGM) vendors, their hardware and mobile app dev cycles are on very different cycles. But to integrate often, they virtualize BLE signals to real devices in the cloud as part of their mobile app test scripts. They also adopt “IoT ninjas” as part of the experience team, hardware/firmware engineers that are in change of prototyping changes on the device side, to make sure that development and testing on the mobile app side is as enabled as possible.
-
Adding IoT to the mix will change your pyramid structure, adding pressure to rely on standards/interfaces as well as manual testing time for E2E scenarios.
-[For more on IoT Testing, see my deck from Mobile/IoT Dev+Test 2017 here]
-Automated Testing Requires Standard Interfaces
-
There are plenty of smart people looking to solve the busy-work problem with writing tests. Facebook Infer, Appdiff, Functionalize, and MABL are just a few of the new technologies that integrate machine learning and AI to reduce time-spend on testing busy-work.
-But any and all programmatic approach, even AI, requires standard interfaces; in our case, universally accepted development AND testing frameworks and technologies.
-Tool ecosystems don’t get built without foundational standards, like HTML/CSS/JS, Android, Java, and Swift. And when they want to innovate on hardware or platform, there will always be some gaps, usually in automation around the new stuff.
-Example Automation Gap: Fingerprint Security
-Unfortunately for those of us who see the advantages of integrating with innovative platform capabilities like biometric fingerprint authentication, automated testing support is scarce.
-What this means is that we either don't test certain critical workflows in our app, or we manually test them. What a bummer to velocity.
-The solution is to have people who know how to implement multiple test frameworks and tools in a way that matches the velocity requirements of development.
-
For more information in this, see my deep-dive on how to use Appium in Android development to simulate fingerprint activities in automated tests. It's entirely possible, but requires experience and a planning over how to integrate a mobile lab into your continuous integration pipeline.
--
Tailoring Fast Feedback to Resources (and vice versa)
-As you incrementally introduce reality into every build, you’ll run into two problems: execution speed and device pool limits.
-To solve the execution speed, most development teams parallelize their testing against multiple devices at once, and split up their testing strategy to different schedules. This is just an example of a schedule against various testing types.
-
For more on this, I published a series of whitepapers on how to do this.
-TL;DR recap
-Automating the quality of our web and mobile apps keeps us accurate, safe, and confident; but isn't easy. Fortunately we have many tools and a lot of thought put in already to how to do this. Notwithstanding ignorance of some individuals, automation continues to change the job landscape over and over again.
-Testing always takes tailoring to the needs of the development process to provide fast feedback. The same is true in reverse: developers need to understand where support gaps exist in test frameworks and tooling, otherwise they risk running the "ship" aground.
-This is why, and my mantra remains, it is imperative to velocity to have the right people in the planning room when designing new features and integrating capabilities across significant technological boundaries.
-Similarly, in my research on developer efficiency, we see that there is a correlation between increased coverage over non-functional criteria on features and test coverage. Greater completeness in upfront planning saves time and effort, it's just that simple.
-Just like Conway’s “law”, the result of your team, it’s structure, communication patterns, functions and dysfunctions, all show up in the final product. Have the right people in the room when planning new features, retros, and determining your own definition of done. Otherwise you end up with more gaps than simply in automation.
-Meta / cliff notes:
--
-
- “Everyone owns quality” means that the whole team needs to be involved in testing strategy
-
-
-
- To what degree are various levels of testing included in Definition of Done? -
- Which test sets (i.e. feedback loops) provide the most value? -
- How are various tests triggered, considering their execution speed? -
- Who's responsible for creating which types of tests? -
- How are team members enabled to interpret and use test result data? -
- When defects do escape certain stages, how is RCA used to close the gap? -
- Who manages/fixes the test execution framework and infrastructure? -
- Does the benefits of the current approach to testing outweigh the cost? -
- - Multiple testing framework / tool / platform is 200 OK
-
-
-
- We already use separate frameworks for separate test types
-
-
-
- jUnit/TestNG (Java) for unit (and some integration) testing -
- Chakram/Citrus/Postman/RestAssured for API testing -
- Selenium, Appium, Espresso, XCTest for UI testing -
- jMeter, Dredd, Gatling, Siege for performance testing -
- - Tool sprawl can be a challenge, but proper coverage requires plurality -
- Don't overtax one framework or tool to do a job it can't, just find a better fit -
- - We already use separate frameworks for separate test types
-
- Incremental doses of reality across architecturally significant boundaries
-
-
-
- We need reality (real devices, browsers, environments) to spot fragility in our code and our architecture -
- Issues tend to clump around architecturally significant boundaries, like API calls, hardware interfaces, and integrations to monolithic components -
- We stub/mock/virtualize to speed development; signs of "significant" boundaries, but it only tells us what happens in isolation -
- A reliable code pipeline can do the automated testing for you, but you still need to tell it what and when to test; have a test execution strategy that considers:
-
-
-
- testing types (unit, component, API, integration, functional, performance, installation, security, acceptance/E2E, ...) -
- execution speed (<2m, <20m, <2h, etc) vs. demand for fast feedback -
- portions of code that are known-fragile -
- various critical-paths: login, checkout, administrative tasks, etc. -
- - Annotations denote tests that relate across frameworks and tools
-
-
-
- @Signup, @Login, @SearchForProduct, @V2Deploy -
- Tag project-based work (like bug fixes) like: JIRA-4522 -
-
- - Have the right people in the room when planning features
-
-
-
- Future blockers like test framework support for new hardware capabilities will limit velocity, so have test engineers in the planning phases -
- Close the gap between what was designed vs. what is feasible to implement by having designers and developers prototype together -
- Including infrastructure/operations engineers in planning reduces later scalability issues; just like testers, this can be a blocker to release readiness -
- Someone, if not all the people above, should represent the user's voice -
-
More reading:
--
-
- On Intelligence, Jeff Hawkins -
- Branch Permissions for your Git repositories -
- Continuous Integration Doesn't Work -
- Why Apps Fail, a Survey of Mobile Users -
- The Test Pyramid - Martin Fowler -
- The Forgotten Layer of the Test Automation Pyramid - Mike Cohen -
- Allistar Scott - The Test Pyramid Ice Cream Cone Anti-pattern -
- Kevin Henney, "Code at Risk" (architectural boundaries), Goto Conference -
- Blockchain on IoT - Fingerprint Security for Your Bitcoin Wallet, FTSafe -
- DON'T PANIC (or how to prepare for IoT with a mature test strategy) -
- Facebook Infer - AI tool squashes bugs with intelligent static code analysis -
- Appdiff - the first API-powered mobile testing platform -
- AI for software testing closes the coverage gap between tests and features -
- MABL - machine intelligence for microservices and apps -
- A Developer's Guide to Build-time Quality - Paul Bruce -
- The Path to Releasing Confidently in DevOps - Paul Bruce, Peter Crocker -
In this article, I'll show how you can use Appium to automate fingerprint authentication on Android mobile devices. The general process also applies to iOS, though specific implementation is not discussed here.
-This is based on work I did in preparation for presenting at Mobile Tea Boston in June 2017. This example is just a small part of a broader conversation on automating quality across the delivery pipeline.
-Git example: https://github.com/paulsbruce/FingerprintDemo
-Fingerprint Security: Great for UX
-
First question I asked was "why would we integrate fingerprint login functionality into our apps?" The short answer is "high security, low friction". There are compelling use cases for fingerprint authentication.
-Paswordless systems usually require people to use SMS or email to confirm login...high friction IMO to the user experience, but who wants their user to leave their UX purposely? This is better security at the cost of poor workflow.
-Multi-factor authentication is another good user case. Using biometric ensures that the unique identity of the individual is presented along with additional credentials.
-Step-up authentication is another popular method of keeping the run-rate user experience frictionless, yet increasing protection over sensitive information and operations on a user's account.
-Fingerprint Security: Bad for Development Velocity
-
So for teams who want to implement fingerprint authentication into their mobile apps, this also means they need to automate tests that integrate fingerprint security. What does the test automation process look like?
-In short, it's a mess. Android libraries and the default UI test framework Espresso contain zero support for fingerprint automation. Since October 2015 with the release of Android 6.0 M, Google provides a standard API for integrating these features into mobile app code, but no way of automating it.
-The same is true for Touch ID on iOS, though there are interactive ways to simulate fingerprint events when running XCTest suites in XCode, there is no easy way of writing an automated test that can provide coverage over these workflows.
-Without some other automation alternative, these portions of functionality fall prey to the ice-cream cone anti-pattern. What a pity.
-Solution: Find the Right Framework
-Espresso is fast because it runs directly alongside the main app code on the device. However, since the only way Google provided us to simulate fingerprint events is through ADB (i.e. 'adb -e emu finger touch ...'), this has to be run on the machine where Android tools are installed and where the device is connected.
-
Appium, an open source outgrowth of Selenium for mobile apps, is architected differently from Espresso and XCTest. Though often slower for this reason, it has some advantages too:
-
Instead of running directly on the device as a sibling process, Appium tests are executed from a server to which the devices are connected. This provides a context whereby we can inject device-specific commands against the device, in combination with the calls through the testing framework itself, to simulate the entire workflow on the device in one script.
-An example of this can be found in my Github FingerprintDemo repo.
-Because I want to write all my code and tests in the same IDE, I keep unit tests and Espresso tests as part of the normal conventions in the 'app' module, but I create a separate module called 'appium' that can be compiled as a separate jar artifact from the main APK. This keeps my Gradle dependencies for testing separate from my app and my build.gradle scripts clean and clear.
-In short, it boils down to test code that looks like this:
-
Appium + fingerprint = depends on your lab
-If you manage a very small local lab, you have the liability control to execute whatever custom commands you need to on your devices.
If you've graduated to using devices (emulators/simulators/real) in the cloud via some service like Firebase, Perfecto, or TestObject, then your ability to simulate fingerprint events reliably really depends on which one you're using.
-
For instance, both Perfecto and TestObject provide SSH direct connections to devices, so in theory you could run custom ADB commands against them; Firebase and AWS Device Farm aren't even close to having this capability.
-In practice, these cloud services also provide automation endpoints and SDKs to execute these tasks reliably. Perfecto, for instance, has both DevTunnel direct access and scripted fingerprint simulation support in Appium.
-Treat Code and Tests as Equal Citizens
-Everyone should have access to app code AND test code. Period. Some large organizations often fear that this will leak proprietary secrets to offshore and out-of-cycle testing teams. That's what contracts and proper repository permissions are for.
-The benefit for modern teams is that test engineers have better visibility into the app, making test creation faster and initial root cause analysis of defects found faster. In my example, this is what the simplified IDE experience looks like:
-
Now that we can press the play button on A) our app, B) our unit and Espresso tests, and C) our E2E fingerprint Appium tests, everyone on the team has the option to make sure their changes don't introduce negative impacts on all aspects of the user experience.
-'Works on My Machine' Isn't Good Enough
-Test code applies first and foremost to the development experience, but also to the build system later on. In the case in including Appium tests in an Android project, this means we need to be keenly aware of the test infrastructure used to simulate fingerprint actions locally against emulators.
-Expect that you will need to “productionize” this process to fit into the build process. By introducing a number of new moving parts (emulators, Appium, custom adb commands) we’ll also need to perpetuate that as a build stack.
-I’m a Jenkins nerd, so what this means in terms of build infrastructure is that we need to create build nodes that contain the components necessary to run Appium tests in isolation of other processes as well. Emulators keep the solution device-independent and can simplify the test execution logistics, but only provide a very narrow slice of reality.
-
To integrate real devices into this mix, you either have to manage a local Appium grid (which again, is challenging) or write your tests to use a cloud lab solution. In the end, you'll have to parameterize your tests along the following environment variables:
--
-
- Appium server address
-
-
-
- localhost for development workstations and Appium emulator stack in CI -
- Shared/cloud host for real devices -
- - (if emulators)
-
-
-
- emulator image (i.e. Nexus_6_API_24, etc.) -
- - Device capabilities
-
-
-
- Platform (Android/iOS) -
- Platform version -
- App (binaries) under test -
- (if shared/cloud) credentials or API keys -
-
Recap:
-Since there's no support for fingerprint simulation directly from Espresso, we have to rely on other test frameworks like Appium to cover these use cases. Really, the test architecture needs to fit the use case, and Appium provides us a way to mix test framework calls with native commands to other mobile tools. This requires us to introduce complexity carefully, plan for how that impacts our build-verification testing stack when triggered by continuous integration.
-More reading:
--
-
- Automating the Quality of Your Digital Front Door - Paul Bruce, June 2017 -
- Why fingerprint security? High-security, low friction. -
- paulsbruce@Github: Fingerprint Demo (Android app) -
- DevOps.com: when testing mobile apps, use a mobile device cloud -
- How to Automate Fingerprint authentication with Perfecto cloud devices -
In theory, a completely Docker-ized version of an Appium mobile UI test stack sounds great. In practice, however, it's not that simple. This article explains how to structure a mobile app pipeline using Jenkins, Docker, and Appium.
-
TL;DR: The Goal Is Fast Feedback on Code Changes
-When we make changes, even small ones, to our codebase, we want to prove that they had no negative impact on the user experience. How do we do this? We test...but manual testing is takes time and is error prone, so we write automated unit and functional tests that run quickly and consistently. Duh.
-As Uncle Bob Martin puts it, responsible developers not only write code that works, they provide proof that their code works. Automated tests FTW, right?
-Not quite. There are a number of challenges with test automation that raise the bar on complexity to successfully getting tests to provide us this feedback. For example:
--
-
- How much of the code and it's branches actually get covered by our tests? -
- How often do tests fail for reasons that aren't because the code isn't working? -
- How accurate was our implementation of the test case and criteria as code? -
- Which tests do we absolutely need to run, and which can we skip? -
- How fast can and must these tests run to meet our development cadence? -
Jenkins Pipeline to the Rescue...Not So Fast!
-Once we identify what kind of feedback we need and match that to our development cadence, it's time to start writing tests, yes? Well, that's only part of the process. We still need a reliable way to build/test/package our apps. The more automated this can be, the faster we can get the feedback. A pipeline view of the process begins with code changes, includes building, testing, and packaging the app so we always have a 'green' version of our app.
-
Many teams chose a code-over-configuration approach. The app is code, the tests are code, server setup (via Puppet/Chef and Docker) is code, and not surprisingly, our delivery process is now code too. Everything is code, which lets us extend SCM virtues (versioning, auditing, safe merging, rollback, etc.) to our entire software lifecycle.
-Below is an example of 'process-as-code' is Jenkins Pipeline script. When a build project is triggered, say when someone pushes code to the repo, Jenkins will execute this script, usually on a build agent. The code gets pulled, the project dependencies get refreshed, a debug version of the app and tests are build, then the unit and UI tests run.
-
Notice that last step? The 'Instrumented Tests' stage is where we run our UI tests, in this case our Espresso test suite using an Android emulator. The sharp spike in code complexity, notwithstanding my own capabilities, reflects reality. I've seen a lot of real-world build/test scripts which also reflect the amount of hacks and tweaks that begin to gather around the technologically significant boundary of real sessions and device hardware.
-A great walkthrough on how to set up a Jenkinsfile to do some of the nasty business of managing emulator lifecycles can be found on Philosophical Hacker...you know, for light reading on the weekend.
-Building a Homegrown UI Test Stack: Virtual Insanity
-We have lots of great technologies at our disposal. In theory, we could use Docker, the Android SDK, Espresso, and Appium to build reusable, dynamic nodes that can build, test, and package our app dynamically.
-Unfortunately, in practice, the user interface portion of our app requires hardware resources that simply can't be executed in a timely manner in this stack. Interactive user sessions are a lot of overhead, even virtualized, and virtualization is never perfect.
-Docker runs under either a hyperkit (lightweight virtualization layer on Mac) or within a VirtualBox host, but neither of these solutions support nested virtualization and neither can pass raw access to the host machine's VTX instruction set through to containers.
-What's left for containers is a virtualized CPU that doesn't support the basic specs that the Android emulator needs to use host GPU, requiring us to run 'qemu' and ARM images instead of native x86/64 AVD-based images. This makes timely spin-up and execution of Appium tests so slow that it renders the solution infeasible.
-
Alternative #1: Containerized Appium w/ Connection to ADB Device Host
-Since we can't feasibly keep emulation in the same container as the Jenkins build node, we need to split out the emulators to host-level hardware assisted virtualization. This approach also has the added benefit of reducing the dependencies and compound issues that can occur in a single container running the whole stack, making process issues easier to pinpoint if/when they arise.
-So what we've done is decoupled our "test lab" components from our Jenkins build node into a hardware+software stack that can be "easily" replicated:
-
Unfortunately, we can no longer keep our Appium server in a Docker container (which would make the process reliable, consistent across the team, and minimize cowboy configuration issues). Even after you:
--
-
- Run the appium container in priviledged mode -
- Mount volumes to pass build artifacts around -
- Establish an SSH tunnel from container to host to use host ADB devices -
- Establish a reverse SSH tunnel from host to container to connect to Appium -
- Manage and exchange keys for SSH and Appium credentials -
...you still end up dealing with flaky container-to-host connectivity and bizarre Appium errors that don't occur if you simply run Appium server on bare metal. Reliable infrastructure is a hard requirement, and the more complexity we add to the stack, the more (often) things go sideways. Sad but true.
-Alternative #2: Cloud-based Lab as a Service
-Another alternative is to simply use a cloud-based testing service. This typically involves adding credentials and API keys to your scripts, and paying for reserved devices up-front, which can get costly. What you get is hassle-free, somewhat constrained real devices that can be easily scaled as your development process evolves. Just keep in mind, aside from credentials, you want to carefully managed how much of your test code integrates custom commands and service calls that can't easily be ported over to another provider later.
-Alternative #3: Keep UI Testing on a Development Workstation
-Finally, we could technically run all our tests on our development machine, or get someone else to run them, right? But this wouldn't really translate to a CI environment and doesn't take full advantage of the speed benefits of automation, neither of which help is parallelize coding and testing activities. Testing on local workstations is important before checking in new tests to prove that they work reliably, but doesn't make sense time-wise for running full test suites in continuous delivery/deployment.
-Alternative #4: A Micro-lab for Every Developer
-Now that we have a repeatable model for running Appium tests, we can scale that out to our team. Since running emulators on commodity hardware and open source software is relatively cheap, we can afford a "micro-lab" for each developer making code changes on our mobile app. The "lab" now looks something like this:
-
As someone who has worked in the testing and "lab as a service" industries, there are definitely situations where some teams and organizations outgrow the "local lab" approach. Your IT/ops team might just not want to deal with per-developer hardware sprawl. You may not want to dedicate team members to be the maintainers of container/process configuration. And, while Appium is a fantastic technology, like any OSS project it often falls behind in supporting the latest devices and hardware-specific capabilities. Fingerprint support is a good example of this.
-The Real Solution: Right { People, Process, Technology }
-My opinion is that you should hire smart people (not one person) with a bit of grit and courage that "own" the process. When life (I mean Apple and Google) throw you curveballs, you need people who can quickly recover. If you're paying for a service to help with some part of your process as a purely economic trade-off, do the math. If it works out, great! But this is also an example of "owning" your process.
-Final thought: as more and more of your process becomes code, remember that code is a liability, not an asset. The less of if, the more lean your approach, generally the better.
-More reading:
--
-
- Automating the Quality of Your Digital Front Door -
- Using Appium to Test Fingerprint Authentication on Android Devices -
- Android Debug Features and Tools chat with Sam Edwards -
- Jenkins on Docker to build Android apps -
Question: Are you 'in the right room'?
-As a part-time practitioner and technical evangelist, one of the most important questions I can ask myself is just that: "Am I in the right room now?"
-If our 'room' is a conversation with the right customer, then read on.
-What is a Technical Evangelist?
-Simply put, an effective technical evangelist is an advocate for customer success in ways that map to and multiply organizational values.
-I recently had an opportunity to depart my full-time position as a developer evangelist at an enterprise-focused company. In search of a new fit, I find that one of the things I miss the most is 'being in the room' on major sales opportunities and product strategy sessions. Insider information about what's going on at [name your target customer] is the life-blood of an organization. It grounds your understanding, your point of view, and your stories.
-Evangelists that simply 'blah blah' are just like practitioner-turned-circuit-speaker. While they might have important insights collected along their journey, the time spent talking to crowds ultimately reduces time for other things like checking in code, being on customer calls, and advising product teams.
-You can tell who the fakes are too, just provide your email to TechWell, BZMedia, or any major tour conference and you'll see the same names pop up over and over again like clockwork. In search of people-as-content, these kinds of marketing-driven machines ultimately produce a Gaussian distribution of hit or miss value to actual practitioners, no matter how hard everyone tries.
-A technical evangelist isn't just a mouth, they are a statement of what your organization cares about. Revenue? Your latest product? Community? It can all be assessed by looking at how people, particularly your evangelists, advocate for your customers.
-What Makes for a Great Technical Evangelist?
-Really good technical evangelism is only 10% talking. You have to know what's really going on with your customers, what they're struggling with, be an exceptional and inquisitive listener, and constantly improve your relationships with folks in your organization. To do that, you need to be a great listener.
-What's the goal of a technical evangelist? Summed up it is:
--'Advocate customer success in such a way that it creates inbound to you and to your organization.'
Why? Without successful customers, chances are your product is vaporware. Slinging a crappy, useless product will always come back to bite you, personally and professionally; and without inbound (people wanting you), you yourself may as well be vaporware too.
-What's Involved in Really Great Technical Evangelism?
-Some of the best technical evangelists I know exhibit the following behaviors:
--
-
- Align field activity (anything in your influence sphere) to the vision of a product that you believe truly helps people -
- Ruthlessly listen to customers and relay important insights to product strategy owners -
- Provide insight publicly by facilitating others' (internal and external) voices -
- Know where your competition fails to satisfy customer's needs and make sure that your org doesn't do the same thing -
- Engage customers via established channels (sales, marketing) and find new, even more effective channels of engagement -
- Collecting information about process, technologies, and problems that customers use to improve -
Why Is 'Being In the Room' Important?
-I've had the opportunity to work with companies that have varying degrees of repeatable success selling to everyone from enterprises to startups. What I've found is that people who are worth their weight in gold are regularly requested to be part of conversations on product strategy, organizational changes, customer success, and especially sales prospects.
-I've been out of the full-time evangelism track for only two months (an eternity for me) and I'm jonesin' for a good, old-fashioned customer visit. There's only so much you can glean from 3 month old curated stories from Capital One on The New Stack, or from GotoConference on Youtube. I've seen speardsheets on how [dominant payroll ISV] organize their test strategies to pinpoint and eradicate poorly performing dev teams. I've build lead-gen tools based on elaborate maturity models from [enterprise ticket vendor] organizations. It's like crack for growth nerds like me.
-But you have to earn your right to be 'in the room'. You have to provide value to multiple people. Sales needs you to advocate for their success on the deal. To do that, you need to translate product impact and vision to customers that they actually need. And you need to bring learning back to product strategy.
-You also need to be involved in an important area of research beyond the needs of your current company...which is why I'm a contributing member of the IEEE 2675 working group on a formal standard for DevOps. I used to think that bodies like ISO and IEEE were bureaucratic bullshit; now I understand how important speaking the same language about healthy and necessary behaviors are, be it ISO/IEC/IEEE 29119, IEEE 1012, or in continuous revision of these resources as the industry evolves.
-Case-in-Point: The Natural Evolution of an Evangelist to Product Owner
-I think that a great technical evangelist has multiple career paths, provided they constantly improve and collect artifacts of their improvement over time. I'm currently working to be ISTQB certified, implementing the crap out of Docker in a mobile app delivery pipeline, working on retainer as a contract load tester, and studying to take my scrum certification. Personal vision dictates your cultivation of options, and focus is just a muscle to exercise with tactics.
-My current career goal is to eventually own a significant revenue stream. Revenue is what organizations, growing or otherwise, understand as value. To accomplish this goal, I'll eventually become a product owner. But on my way, I've been learning the various dynamics and tactics of how to scale a technology firm, namely how to interface with sales, marketing, and product strategy.
-Product Managers and Owners who are actively engaged in sales and customer success engagements with customers have a line of sight to exactly what they need. Also, their job title (often earned) can often act as a carrot for sales to dangle when the opportunity is...more complicated...than the usual deal. The value they often bring is to quickly drill in to the most pressing issue that the customer faces and align the prospect to how the product enables them to overcome that challenge.
-Sales Professionals (i.e. 'account managers', 'customer representatives') who are really good follow a simple mantra: 'don't waste anyone's time'. In the past year, I've witnessed a global SVP of Sales perpetually inspire an organization to this end; countless sessions and references to learning materials like Grit, Let's Get Real or Let's Not Play', and other resources that ruthlessly focus on efficient, productive conversations.
-Marketing...what can I say: in an world inundated by information-overload, finding the most effective path to (each of your) audiences is an important part of exercising your Product Owner muscles. Yes, I want to work more directly with development teams, but code is only one component of getting people to use and love your product. Orchestrating go-to-market strategy connects me to everyone: product owners, developers, documentarians, strategists, customers, sales, and leadership. It is a nexus I accept and apparently do well in.
-Should You Hire (or Become) an Evangelist?
-I'm biased, but definitely yes. If you want to close deals, look awesome, increase influence, drive revenue growth, understand your customer better, grow personally and professionally, and ultimately make people so happy that they write great things about you on their own time, then yes, find a good technical evangelist and embed them in the middle of your business.
-If you're looking for one, you can find me on all the usual channels.
- diff --git a/_posts/2017-08-14-performance-is-still-a-feature-not-a-test.html b/_posts/2017-08-14-performance-is-still-a-feature-not-a-test.html deleted file mode 100644 index 41faaa8..0000000 --- a/_posts/2017-08-14-performance-is-still-a-feature-not-a-test.html +++ /dev/null @@ -1,71 +0,0 @@ ---- -layout: post -title: Performance Is (Still) a Feature, Not a Test! -date: 2017-08-14 12:54:53.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- automation -- DevOps -- performance -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '759' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2017/08/performance-is-still-a-feature-not-a-test/" ---- -Since I presented the following perspective at APIStrat Chicago 2014, I've had many opportunities to clarify and deepen it within the context of Agile and DevOps development:
--It's more productive to view system performance as a feature than to view it as a set of tests you run occasionally.
The more teams I work with, the more I see how performance as a critical aspect of their products. But why is performance so important?
-'Fast' Is a Subconscious User Expectation
-Whether you're building an API, an app, or whatever, its consumers (people, processes) don't want to wait around. If your software is slow, it becomes a bottleneck to whatever real-world process it facilitates.
-Your Facebook feed is a perfect example. If it is even marginally slower to scroll through it today than it was yesterday, if it is glitchy, halty, or jenky in any way, your experience turns from dopamine-inducing self-gratification to epinephrine fueled thoughts of tossing your phone into the nearest body of water. Facebook engineers know this, which is why they build data centers to test and monitor mobile performance on a per-commit basis. For them, this isn't a luxury; it's a hard requirement, as it is for all of us whether we choose to address it or not. Performance is everyone's problem.
-Performance is as critical to delighting people as delivering them features they like. This is why session abandonment rates are a key metric on Cyber Monday.
-'Slow' Compounds Quickly
-Performance is a measurement of availability over time, and time always marches forward. Performance is an aggregate of many dependent systems, and even just one slow link can cause an otherwise blazingly fast process to grind to a halt long enough for people to turn around and walk the other way.
-Consider a mobile app; performance is everything. The development team slaves over which list component scrolls faster and more smoothly, spends hours getting asynchronous calls and spinners to provide the user critical feedback so that they don't think the app has crashed. Then a single misbehaving REST call to some external web API suddenly slows by 50% and the whole user experience is untenable.
-The performance of a system is only as strong as it's weakest link. In technical terms, this is about risk. You at least need to know the risk introduced by each component of a system; only then can you chose how to mitigate the risk accordingly. 'Risk' is a huge theme in ISO 29119 and the upcoming IEEE 2675 draft I'm working on, and any seasoned architect would know why it matters.
-Fitting Performance into Feature Work
-Working on 'performance' and working on a feature shouldn't be two separate things. Automotive designers don't do this when they build car engines and performance is paramount throughout even the assembly process as well. Neither should it be separate in software development.
-However, in practice if you've never run a load test, tracked power consumption of a subroutine or analyzed aggregate results, it will be different than building stuff for sure. Comfortability and efficiency come with experience. A lack of experience or familiarity doesn't remove the need for something critical to occur; it accelerates the need to ask how to get it done.
-A reliable code pipeline and testing schedule make all the difference here. Many performance issues take time or dramatic conditions to expose, such as battery degradation, load balancing, and memory leaks. In these cases, it isn't feasible to execute long-running performance tests for every code check-in.
-What does this mean for code contributors? Since they are still responsible for meeting performance criteria, it means that they can't always press the 'done' button today. It means we need reliable delivery pipelines to push code through that checks its performance pragmatically. As pressure to deliver value incrementally mounts, developers are taking responsibility for the build and deployment process through technologies like Docker, Jenkins Pipeline, and Puppet.
-It also means that we need to adopt a testing schedule that meets the desired development cadence and real-world constrains on time or infrastructure:
--
-
- Run small performance checks on all new work (new screens, endpoints, etc.) -
- Run local baselines and compare before individual contributors check in code -
- Schedule long-running (anything slower than 2mins) performance tests into pipeline stage after build verification in parallel -
- Schedule nightly performance regression checks on all critical risk workflows (i.e. login, checkout, submit claim, etc.) -
How Do You Bake Performance Into Development?
-While it's perfectly fine to adopt patterns like 'spike and stabilize' on feature development, stabilization is a required payback of the technical debt you incur when your development spikes. To 'stabilize' isn't just to make the code work, it's to make it work well. This includes performance (not just acceptance) criteria to be considered complete.
-A great place to start making measurable performance improvements is to measure performance objectively. Every user story should contain solid performance criteria, just as it should with acceptance criteria. In recent joint research, I found that higher performing development teams include performance criteria on 50% more of their user stories.
-
In other words, embedding tangible performance expectations in your user stories bakes performance in to the resulting system.
-There are a lot of sub-topics under the umbrella term "performance". When we get down to brass tacks, measuring performance characteristics often boils down to three aspects: throughput, reliability, and scalability. I'm a huge fan of load testing because it helps to verify all three measurable aspects of performance.
-Throughput: from a good load test, you can objectively track throughput metrics like transactions/sec, time-to-first-byte (and last byte), and distribution of resource usage (i.e. are all CPUs being used efficiently). These give you a raw and necessarily granular level of detail that can be monitored and visualized in stand-ups and deep-dives equally.
-Reliability: load tests also exercise your code far more than you can independently. It takes exercise to expose if a process is unreliable; concurrency in a load test is like exercise on steroids. Load tests can act as your robot army, especially when infrastructure or configuration changes push you into unknown risk territory.
-Scalability: often, scalability mechanisms like load balancing, dynamic provisioning, and network shaping throw unexpected curveballs into your user's experience. Unless you are practicing a near-religious level of control over deployment of code, infrastructure, and configuration changes into production, you run the risk of affecting real users (i.e. your paycheck). Load tests are a great way to see what happens ahead of time.
--
Short, Iterative Load Testing Fits Development Cycles
-I am currently working with a client to load test their APIs, to simulate mobile client bursts of traffic that represent real-world scenarios. After a few rounds of testing, we've resolve many obvious issues, such as:
--
-
- Overly verbose logs that write to SQL and/or disk -
- Parameter formats that cause server-side parsing errors -
- Throughput restrictions against other 3rd-party APIs (Google, Apple) -
- Static data that doesn't exercise the system sufficiently -
- Large images stored as SQL blobs with no caching -
We've been able to work through most of these issues quickly in test/fail/fix/re-test cycles, where we conduct short all-hands sessions with a developer, test engineer, and myself. After a quick review of significant changes since the last session (i.e. code, test, infrastructure, configuration), we use BlazeMeter to kick of a new API load test written in jMeter and monitor the server in real-time. We've been able to rapidly resolve a few anticipated, backlogged issues as well as learn about new problems that are likely to arise at future usage tiers.
-The key here is to 'anticipate iterative re-testing'. Again I say: "performance is a feature, not a test". It WILL require re-design and re-shaping as the code changes and system behaviors are better understood. It's not a one-time thing to verify how a dynamic system behaves given a particular usage pattern.
-The outcome from a business perspective of this load testing is that this new system is perceived to be far less of a risky venture, and more the innovation investment needed to improve sales and the future of their digital strategy.
-Performance really does matter to everyone. That's why I'm available to chat with you about it any time. Ping me on Twitter and we'll take it from there.
diff --git a/_posts/2017-09-01-wrangling-promises-in-node-js-3-misconceptions-resolved.html b/_posts/2017-09-01-wrangling-promises-in-node-js-3-misconceptions-resolved.html deleted file mode 100644 index 68e7aff..0000000 --- a/_posts/2017-09-01-wrangling-promises-in-node-js-3-misconceptions-resolved.html +++ /dev/null @@ -1,157 +0,0 @@ ---- -layout: post -title: 'Wrangling Promises in Node.js: 3 Misconceptions Resolved' -date: 2017-09-01 16:07:57.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- Node.js -tags: -- concurrency -- Javascript -- Promises -meta: - _oembed_271f27c092f0da0445e20077cf2e37a6: "{{unknown}}" - _edit_last: '1' - _thumbnail_id: '765' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2017/09/wrangling-promises-in-node-js-3-misconceptions-resolved/" ---- -ES6 (i.e. proper Javascript) isn't the first time Promises were introduced, but formally supports them now. Believe me, you want to start using Promises in your Node.js scripts, but if you're new to the pattern, it can be tricky to get your head around. That's what I hope to help you do in the next 5 minutes.
-What Are Javascript/ES6 Promises?
-My way of explaining it is that Promises are chaining pattern, a convention that helps decouple your code blocks from your execution pattern. Promises can dramatically improve your approach to asynchronous programming (such as how Node.js 8+ prefers) and simplify your callbacks by helping you express them in a linear fashion.
-[caption id="attachment_762" align="alignnone" width="801"]
from Promise (object) on MDN web docs[/caption]
The really easy thing about them is that a Promise ends in either of two conditions:
--
-
- Become fulfilled by a value -
- Become rejected with an exception -
Consider the following code example:
-var fetchJSON = function(url) {
- return new Promise((resolve, reject) => {
- request({ // synchronous call wrapped in Promise
- url: url,
- json: true
- }, function (error, response, body) {
- if(!error && response.statusCode === 200)
- resolve(body); // transformed to an object via 'json: true' above
- else
- reject(error ? error : new Error('Was not 200 OK: ' + body));
- })
- });
-}
-In the above example, the 'fetchJSON' function returns a Promise, not the result of executing the request. Expressing things this way allows us to execute the code immediately, or as part of an asynchronous chain, such as:
-// begins to execute immediately, but results come back asynchronously
-fetchJSON('http://paulsbruce.io/wp-json/wp/v2/tags')
- .then(jsonObj => {
- console.info(jsonObj.map(it => { // print the array of values
- return jp.value(it,"$..name"); // for each tag, value is 'name'
- }));
- })
- .catch(error => {
- console.error(error);
- });
-console.log('after fetchJSON call')
-What's the alternative? Well...I hesitate to show you (the interweb loves to copy and paste) because we would have to:
--
-
- express every asynchronous action as a callback function (which is bulky) -
- indent/embed blocks in a recursive step pattern -
- chain commands by calling the next function from our executing function -
So far, I've made a career of learning how to stand up and say 'I will not build that'. We should do that more often #facebook and you should read this.
-The 3 Misconceptions You Want to Immediately Overcome
-Amongst many I had while learning to use Promises, these are the top three I and most others often struggle to overcome:
--
-
- You can't mix synchronous/callback-oriented code with Promise-based code -
- It's okay to ignore catching errors because it's optional to the Promise chain -
- There's no way to join parallel executing paths of asynchronous Promises -
I focus on these 3 misconceptions because when they're not in your head, you can focus on the simplicity of Promise code. When writing, ask yourself: "is the code I just wrote elegant?" If the answer is no, chances are you're getting hung up on a misconception.
-Mixing Synchronous/Callback-Oriented Code with Promises
-You CAN inject legacy synchronous code (code that doesn't emit Promises), but you have to handle Promise-related tie-ins manually. The code example in the last section does exactly that with the 'request' function. However you DO have to wrap it in a function/lambda that eventually calls the 'resolve' or 'reject' handlers.
-For instance, in a recent integration to Twitter, their 'stream' function does not return a Promise and only provides a callback mechanism, so I wrapped it to resolve or reject:
-async function createTwitterStream(ctx) {
- return new Promise(function(resolve, reject) {
- try {
- client.stream("user", { }, function(stream) {
- resolve(stream);
- stream.on("data", function(event) {
-
- q.push(function() {
- onTwitterEvent(event, ctx);
- });
-
- });
- stream.on("error", function(error) {
- console.error(error)
- throw error;
- });
- });
- } catch(err) {
- reject(err);
- }
- })
-}
-I decided to 'Promisify' this functionality because I wanted to wrap this logic in a Promise-based retry mechanism so that if the initial stream negotiation failed, it would only fail out of the entire script when multiple attempts failed. I opted to pull in the 'promise-retry' package from npmjs. Simplified the calling code dramatically:
- // retry twitter stream up to 5 times
- promiseRetry(function (retry, number) {
- console.info('attempt number', number);
-
- return createTwitterStream(ctx)
- .catch(function(err) {
- console.log(err)
- if(number <= 5) retry(err);
- throw err
- });
- })
- .catch(err => {
- console.log('Epic failure to establish Twitter stream after multiple attempts.')
- throw err;
- });
-
-Can you see how powerful Promises are now? Imagine how coupled the retry code would be with the stream initialization logic. Again, not going to show you what it looked like before for fear of the copy-n-paste police.
-Don't Ignore Error Catching Simply Because the Code Validates!
-At first, as I was re-writing blocks of code to Promise-savvy statements, I was getting a lot of these errors:
-
The problem was that I didn't have '.catch' statements anywhere in the Promise chain. Node.js was interpreting the code as valid until runtime when the error occurred. Bad. Really bad of me. Glad that Node 8 was warning me.
-You don't have to write '.catch' after every Promise, particularly if you're returning Promises through functions, so long as the error is handled in at least one place up the Promise chain hierarchy. The Promise model provides you granularity on which errors you want to bubble up.
-For instance, in the above code, I DON'T bubble up individual event/tweet errors, but I DO allow stream initialization errors to bubble up to the calling retry code. I can also selectively extend the individual stream event errors to become a bigger problem if the message back from twitter is something like '420 Enhance Your Calm' which essentially means "back the fuck off, asshole".
-You CAN Join/Wait for Parallel Executing Promises
-The Promise chain lets us string together as many sequential steps as we want via the '.then' handler. But what about waiting for parallel threads of code?
-Using the 'Promise.all' function, we can execute separate Promises asynchronously in parallel to each other, but wait in a parent async function by prefixing with the 'await' statement. For example:
- await Promise.all([
- loadKeywords(ctx)
- .then((kwds) => {
- ctx.keywords = kwds;
- console.debug("Keywords: " + ctx.keywords);
- })
- ,
- loadFriendlies(ctx)
- .then((frnds) => {
- ctx.friendlies = frnds;
- console.debug("Friendlies: " + ctx.friendlies);
- })
- ]);
-
- console.debug("Initialization complete. Processing events.");
-Within an async function, the above code will wait for both Promises to complete before printing the final statement to the console.
-I can tell, now you're really getting the power of decoupling code expression from code execution. I told you that you'd want to start using Promises. As such, I suggest reading up on the links at the end of the article.
-Hidden Lesson: Don't Bury Your Head in the Sand!
-My takeaway from all this learning is that I should have been applying lessons learned in my Java 7 work to other areas like Node.js. Promises isn't a new idea (i.e. Java Futures, Async C#). If a pattern emerges in one language or framework, it's very likely to already exist in others. If not, find people and contribute to the solution yourself.
-If you run into issues, ping me up on Twitter or LinkedIn, and I'll do my best to help in a timely manner.
-More reading:
--
-
- ES6 Promises, Dave Atchley -
- Javascript Promises, an Introduction -
- Intro to async Javascript, an Incremental Tutorial to Promises -
- "I will not build this" - Mike Monteiro - In Praise of Ordinary People -
- Promise Pipelineing: how expressing your code differently can reduce latency -
This week, I've been exploring the InfluxData tech stack. As a muse, I decided to move some of my social media sharing patterns formal algorithms. I also want to use my blog as a source for keywords, filter out profanity, and apply sentiment analysis to clarify relevant topics in the tweets.
-Github repo for this article: github.com/paulsbruce/InfluxTwitterExample
-What Is InfluxData?
-Simply put, it's a modern engine for metrics and events. You stream data in, then you have a whole host of options for real-time processing and analysis. Their platform diagram speaks volumes:
-[caption id="attachment_768" align="alignnone" width="840"]
From InfluxData.com Telegraph Overview[/caption]
Based on all open source components, the InfluxData platform has huge advantages over other competitors in terms of extensibility, language support, and its community. They have cloud and enterprise options when you need to scale your processing up too.
-For now, I want to run stuff locally, so I went with the free sandbox environment. Again, completely open source stack bits, which is very cool of them as lots of their work ends up as OSS contributions into these bits.
-Why Process Twitter Events in InfluxDB?
-Well, frankly, it's an easy source for real-time data. I don't have a 24/7 Jenkins instance or pay-for stream of enterprise data flowing in right now, but if I did, I would have started there because they already have a Jenkins data plugin. :)
-But Twitter, just like every social media platform, is a firehose of semi-currated data. I want to share truly relevant information, not the rest of the garbage. To do this, I can pre-filter based on keywords from my blog and 'friendlies' that I've trusted enough to re-share in the past.
-The point is not to automatically re-share content (which would be botty), but to queue up things in a buffer that would likely be something I would re-tweet. Then I can approve or reject these suggestions, which in turn can be a data stream to improve the machine learning algorithms that I will build as Kapacitor user-defined functions later on.
-Streaming Data Into InfluxDB
-There's a huge list of existing, ready-to-go plugins for Telegraph, the collection agent. They've pretty much thought of everything, but I'm a hard-knocks kind of guy. I want to play with the InfluxDB APIs, so for my exploration I decided to write a standalone process in Node.js to insert data directly into InfluxDB.
-To start, let's declaring some basic structures in Node to work with InfluxDB:
-const dbname = "twitter";
-const measure = "tweets";
-const Influx = require("influx");
-const influx = new Influx.InfluxDB({
- host: "localhost",
- database: dbname,
- schema: [
- {
- measurement: measure,
- fields: {
- tweetid: Influx.FieldType.INTEGER,
- relevance: Influx.FieldType.FLOAT,
- user: Influx.FieldType.STRING,
- volatile: Influx.FieldType.BOOLEAN,
- raw: Influx.FieldType.STRING
- },
- tags: [
- "keywords"
- ]
- }
- ]
-});
--
-
- dbname: the InfluxDB database to insert into -
- measure: the measurement (correlates to relational table) to store data with -
- fields: the specific instance data points to collect on every relevant Tweet -
- tags: an extensible list of topic-based keywords to associate with the data -
Making Sure That the Database Is Created
-Of course, we need to ensure that there's a place and schema for our Twitter data points to land as they come in. That's simple:
-influx.getDatabaseNames()
- .then(names => {
- if (!names.includes(dbname)) {
- return influx.createDatabase(dbname);
- }
- });
-Saving Pre-screened Tweets as InfluxDB Data Points
-Minus the plumbing of the Twitter API, inserting Tweets as data points to InfluxDB is also very easy. We simply need to match our internal data structure to than of the schema above:
-// after processing, save Tweet result to InfluxDB directly
-function saveTweetToInflux(result) {
- influx.writePoints([
- {
- measurement: "tweets",
- tags: { // array of matched keywords
- keywords: (result.tags.length > 0 ? result.tags.join(",") : [])
- },
- fields: {
- tweetid: result.tweetid, // unique tweet id
- relevance: result.relevance, // 0.0 to 1.0 for later analysis
- user: result.user, // original twitter user
- volatile: result.volatile, // contains blocked words?
- raw: JSON.stringify(result) // complete tweet for later analysis
- },
- }
- ]).catch(err => {
- console.error(`Error saving data to InfluxDB! ${err.stack}`);
- });
-}
-Notice that the keywords (tags) can be a simple Javascript array of strings. I'm also optionally inserting the raw data for later analysis, but aggregating some of the most useful information for InfluxQL queries as fields.
-The InfluxDB Node.js client respects ES6 Promises, as we can see with the '.catch' handler. Huge help. This allows us to create robust promise chains with easy-to-read syntax. For more on Promises, read this article.
-Verifying the Basic Data Stream
-To see that the data is properly flowing in to the InfluxData platform, we can use Chronograf in a local sandbox environment and build some simple graphs:
-
To do this, we use the Graph Editor to write a basic InfluxQL statement:
-
SELECT count("tweetid") AS "count_tweetid"
-FROM "twitter"."autogen"."tweets"
-WHERE time > :dashboardTime:
-GROUP BY time(5m), "keywords"
-The simple graph shows a flow of relevant tweets grouped by keyword so we can easily visualize as real-time data comes in.
-A Few Ideas and Next Steps
-Of the many benefits of processing data on the InfluxData platform, processing in Kapacitor seems to be one of the most interesting areas.
-Moving forward I'd like to:
--
-
- Move Sentiment Analysis with Rosette from Node into Kapacitor -
- Add Machine Learning into Kapacitor for
-A) clarifying relevance of keywords based on sentiment entity extraction
-B) extract information about the positivity / negativity of the tweet
- - Catch high-relevance notifications and send to Buffer 'For Review' queue
-A) accepts and rejects factor back into machine learning algorithm
-B) follow-up statistics about re-shares further inform ML algorithm
- - Have Kapacitor alert when:
-A) specific high-priority keywords are used (use ML based on my tweets)
-B) aggregate relevance for a given keyword spikes (hot topic)
-C) a non-tracked keyword/phrase is used in multiple relevant tweets
-(could be a related topic I should track, event hashtag, or something else)
-
As You Build Your Own, Reach Out!
-I'm sure as I continue to implement some of these ideas, I'll probably need help. Fortunately, Influx has a pretty active and helpful Community site. Everything from large exports, plugin development, and IoT gateways are discussed there. Jack Zampolin, David Simmons, and even CTO Paul Dix are just a few of the regular contributors to the conversation over there.
-And as always, I like to help. As you work through your own exploration of InfluxData, feel free to reach out via Twitter or LinkedIn if you have comments, questions, or ideas.
diff --git a/_posts/2017-09-17-devops-is-organizational-operational-and-orthogonal.html b/_posts/2017-09-17-devops-is-organizational-operational-and-orthogonal.html deleted file mode 100644 index 6a3c433..0000000 --- a/_posts/2017-09-17-devops-is-organizational-operational-and-orthogonal.html +++ /dev/null @@ -1,69 +0,0 @@ ---- -layout: post -title: DevOps is Organizational, Operational, and Orthogonal -date: 2017-09-17 15:33:06.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- DevOps Days Boston 2017 -tags: -- IEEE 2675 -meta: - _edit_last: '1' - _thumbnail_id: '789' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2017/09/devops-is-organizational-operational-and-orthogonal/" ---- -Some people seem to think that DevOps is a buzzword. It is not. At all.
-DevOps Is an Organizational Paradigm
-----If you want to go fast, go alone. If you want to go far, go together.
-
DevOps Is an Operational Paradigm
-DevOps Is an Orthogonal Paradigm
--
-
- How can you go "faster" if your acquisition process takes many months? -
- How can you go "faster" if a supplier doesn't provide a way to validate that you integrated their product or service correctly? -
- How can testing be continuous if it isn't automated and therein scalable? -
- How can you expect marketing to crush numbers if you don't integrate them into your sprints (their work often needs weeks/months of lead time)? -
- If your onboarding process doesn't train new engineering recruits (dev/test/ops/PM/IT) on your lifecycle, how can you expect them to "go fast"? -
- If it takes days/weeks for customer feedback to reach development cycles, how can you expect to be building the "right thing" tomorrow? -
DevOps Is Just a Word
-[wonderplugin_slider id="1"]
diff --git a/_posts/2017-09-18-no-root-cause-in-emergent-behavior-devops-days-boston-2017.html b/_posts/2017-09-18-no-root-cause-in-emergent-behavior-devops-days-boston-2017.html deleted file mode 100644 index 3dc5400..0000000 --- a/_posts/2017-09-18-no-root-cause-in-emergent-behavior-devops-days-boston-2017.html +++ /dev/null @@ -1,66 +0,0 @@ ---- -layout: post -title: No Root Cause in Emergent Behavior - DevOps Days Boston 2017 -date: 2017-09-18 20:34:41.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- DevOps Days Boston 2017 -tags: -- DevOps Days Boston 2017 -- emergent behavior -- systems thinking -meta: - _edit_last: '1' - _thumbnail_id: '817' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2017/09/no-root-cause-in-emergent-behavior-devops-days-boston-2017/" ---- -At DevOpsDays Boston 2017, Matthew Boeckman presented on how emergent behavior in complex systems requires us to re-think our root cause analysis paradigms. His slides are here. I also had a great time talking meta with Matthew afterhours, but that's for a later post.
-Traditional RCA in Complex Systems
-Unfortunately, traditional RCA focuses on what and who. Despite its roots stemming from NASA, in the software world, RCA is misaligned to find only one channel of causality. A fishbone diagram shows this:
-
This might be okay for simple systems (i.e. 3-tier web/app/data servers). There's much more to this: networking, hosting, and operating environments. Beyond that, users access in both benign and benevolent ways.
-Waterfall encouraged us to minimize complexity by locking down state (i.e. promote a "don't change" mentality). Waterfall (think 12mo cycles) encourages us to think that change is the developer's fault. And there were a lot of constrains in the 80s and 90s, most of them are no longer true.
-
Root cause is fine for static models, but there are bad when it comes to "lots of boxes", cloud-based dynamic and distributed systems. It's very hard to trace the source of problems in this new world. Change vectors (a/b testing, reconfigurations, migrations, feature flags) abound, in fact they're encouraged.
-Our systems are far more complex than they were 20 years ago. They involve the whole stack, the whole team, and the whole organization.
-
Paul's Take: Occam's Anti-Razor
-A heuristic idea we often employ is Occam's razor, in general that, the simplest answer is often the right one. Coupled with a confirmation bias, we (humans) often look for a single causal root to the problems we see. Then we build processes that inherit our bias. But what if operational failures occur because of multiple causes, chain reactions that exceed the typical '5 whys' RCA model?
-As quickly as the concept of the razor was introduced, Chatton, a contemporary, countered the idea with: "If three things are not enough to verify an affirmative proposition about things, a fourth must be added, and so on." Similarly, many ascribe a balance of simplicity and complexity in solving problems to the quote "Make things as simple as possible, but no simpler." by Einstein.
-The idea is right fit...right fit of simplicity/complexity to the problem at hand. With complex systems, we can't always assume that the simple answer is the most useful one in future scenarios.
-Our Systems Aren't Trees, They're Forests
-Emergence is about collective behaviors, systems we connect and integrate over time, and not simply the aggregate of behaviors emitted by individual subcomponents and nodes.
-We need to develop, test, deploy, monitor and issue resolve them like the complex semi-organic systems they are, part of an ecosystem of services and fallible subsystems that they are. We can no longer afford to ignore better paradigms for dealing with them.
-Enter Systems Thinking. Understanding why things emerge takes more than an ops dashboard and intuition. Sometimes analysis on complex problems requires a multi-variate perspective.
-Paul's advice: System's thinking is a much broader topic that, if you haven't actually studied, it would serve you well to listen to The Fifth Discipline by Peter Senge. As context for my presentation in April on IoT testing, this made me realize that systems thinking was a necessary mental tool moving forward.
-Systems thinking helps us to identify activities, interactions, and ultimately change vectors contributing to emergent behaviors. Understanding which dials and levers are involved in the problem enables later actions to resolve the issue. This feeling of being at home in the problem space is also similar to "cynefin", a welch/gaelic term that in Scottish (my heritage!) means:
-"a place to live and belong. where the nature of what's around you feels right and welcoming"
-Not at all coincidentally, the Cynefin framework as applied to emergent behavior helps us make quick decisions during and about incident management situations.
-Staying Ahead of Emergent Behavior
-The fact is that most workforces, small or large, are a revolving door. So is your current system state after multiple releases and infrastructure migrations. There be the monsters. Software is dynamic, and so should be your product discovery process, your learning loops, your incident management model, and so on.
-The Cynefin framework gives us this quadrant visual to show that various issues need to be addressed differently:
-
The fact is, each of these quadrants assume two things:
--
-
- The issue occurred already, so you need to fix it and learn from it -
- Information needs to be radiated (sensed) to make "sense" of it -
In my after-hours chat with Matthew, we dived into the issue of metrics. Measuring issue tracking goes beyond mean-time-to-resolution (MTTR). Issues that are flagged with *how* they were resolved using Cynefin categories now have an opportunity for improvement.
-Paul's Take: could this be a JIRA custom field? Just thinking out loud.
-Tracking the delta on a specific issue (what approach someone thought should be used at first vs. what would have been better after the fact) is a way to measure successfulness and improvement on a spot basis.
-Then over time, aggregates can be used to show team and organizational reflexiveness to dynamic, emergent behavior. Though neither of us have customer anecdotes or proof-of-concept clients, I challenge you who are reading this to try it out for a few sprints or whatever intervals you use.
-Summary
-We need to embrace emergent behavior and learn how to approach incidents better using systems thinking and frameworks like Cynefin. Unlike traditional RCA, we'll need to step out of our comfort zones, see what works, and learn from our mistakes.
-Matthew is a Denver, Colorado native, and has spoken at other conferences like Gluecon (wicked!). If you have questions, ping him (and me) up on Twitter and let's get a dialog going.
- -[wonderplugin_slider id="1"]
diff --git a/_posts/2017-09-19-enterprise-wild-west-at-devops-days-boston.html b/_posts/2017-09-19-enterprise-wild-west-at-devops-days-boston.html deleted file mode 100644 index daedfea..0000000 --- a/_posts/2017-09-19-enterprise-wild-west-at-devops-days-boston.html +++ /dev/null @@ -1,77 +0,0 @@ ---- -layout: post -title: Enterprise Wild West - DevOps Days Boston 2017 -date: 2017-09-19 06:30:21.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- DevOps Days Boston 2017 -tags: -- DevOps Days Boston 2017 -meta: - _oembed_de8c8550ae01352b64e2d3a86b6a3f60: "{{unknown}}" - _edit_last: '1' - _thumbnail_id: '804' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2017/09/enterprise-wild-west-at-devops-days-boston/" ---- -Rob Cummings' keynote at DevOps Days Boston 2017 explored how Simon Wardley’s Pioneers, Settlers, and Town Planners model applies in enterprise engineering and large organizations. The general idea is:
--
-
- Pioneers: explorers of new ideas, create prototypes, prove the need -
- Settlers: stealers of new ideas, move prototypes to MVP, prove feasibility -
- Town Planters: manufacturers, MVPs to industries, prove scalability -
[caption id="attachment_792" align="alignnone" width="1912"]
Bits or pieces?: On Pioneers, Settlers, Town Planters and Theft[/caption]
The Problem: Overly Simplistic Approaches
-Bi-modal IT splits the org into Mode 1 (systems of record) and Mode 2 (systems of innovation). Mode 1 has less line of sight to customers and is governed by enterprise architecture and governance. Mode 2 often runs into Mode 1 when .... The problem is that often, there's no flow between Mode 1 and Mode 2. Bi-modal is overly simplistic.
-
The book "Thinking in Systems" is a great place to start your journey beyond these modes. Transition states and feedback loops exist already in your org, but realizing where they are and how they could be improved takes practice and group engagement.
-Paul's advice: System's thinking is a much broader topic that, if you haven't actually studied, it would serve you well to listen to The Fifth Discipline by Peter Senge. As context for my presentation in April on IoT testing, this made me realize that systems thinking was a necessary mental tool moving forward.
-Everyone Innovates...Sometimes.
-Pioneers live outside standards, fail often, and don't necessarily make decisions based on metrics. Find the new horizon. That's how they innovate, they bring ideas from outside in.
-Settlers make prototypes real, building trust in the org, kick off ecosystems around the adoption of ideas, but sometimes suffer from adoption problems. They bring ideas further in to the org.
-Town Planners focus on ops efficiency, build services and platforms that Pioneers rely on for future innovations. They're metrics heavy and bring reality to the operation of ideas.
-Fostering Friendly Theft
-The Wild West is a "theft-based pull model". There are no mandates. Theft occurs from right to left (pioneers on the left). re-use from left to right.. This is a good thing. Everyone is excellent and everyone both should participate in empathy. Foster feedback loops and maintain pull culture.
-
The Wild model exists within a team, not as separate departments. Again, for DevOps we're not talking about traditional cost centers and departments; we've got mixed teams that are aligned on a shared goal with their own perspectives on how to do things best, together.
-Paul's Take: DevOps Requires Buy-in from Everyone
-For DevOps to work, a team needs to understand and adapt to their organizational ecosystem. So while the micro-mechanics of the Wild West help us pull new ideas in on a continual basis, there has to be an understanding that extends across the whole org.
-Many conversations at DevOps Days Boston 2017 on day 1 expressed the need for "buy-in from the top", but effective DevOps also requires buy in from everyone. Teams need to align the virtues of DevOps to how they can positively impact the organization. It does no good for an SMB VP of Engineering to apply DevOps if the purpose of doing so hasn't been clearly articulated in terms that other dependencies (like the developers, operations, sales, marketing, finance, and support) understand. But when you do so, it's much easier to carry people with you in planning and execution.
-DevOps is Organizational, Operational, and Orthogonal. Applying it in isolation only decreases the value it brings to us.
-Scaling to the Enterprise
-Rob shared an anonymized anecdote from a large company where the Wild West model was adopted:
-A small group of pioneers realized "we need to fix this, can't meet customer needs". They knew how to do it and got CIO sponsorship. The team got to MVP status with code. Unfortunately, the Wild West model was not immediately adopted beyond that initial release.
-"We were trying to push the model onto the team." Even though everything done up to that point focused on ease-for-enterprise (weekly demos, code was open sourced, process transparency), adoption took time.
-Eventually, another team took the ideas and model, shipped their thing to production, then other teams followed. "Now we have a 'proliferation problem'...people started customizing tools and artifacts." Teams often stuck with some favorite tools, and in DevOps culture, tailoring is huge.
-But not everyone wants to build their own house. For example, code pipelines...yuk. So Planners came in and built a commodity pipeline platform. This requires talent, people who have skill and can scale, understand operational efficiency.
-Summary
-Here are a few anti-patterns that will reduce friction and increase your flow.
--
-
- Using enterprise architecture to prevent waste and force adoption.
-Don't use it as a gate to get to production!
-.
- - Relying on innovation labs or CoE for pioneers.
- Teams outside your org toss things in that often don't work inside the org. Be super-public so settlers are likely to steal. Change CoE to "Center of Practice", inclusive, then everyone can be excellent.
-.
- - Don't forget that your org requires a systems thinking approach.
-Create flows not barriers. Each role is filled with excellent people.
-
-
More reading:
--
-
- Rob's DevOps Bookshelf -
- DevOps is Organizational, Operational, Orthogonal -
- The Wild West Model: Attitude and Aptitude -
- Org Hacking: Pioneers, Settlers, and Town Planters -
[wonderplugin_slider id="1"]
diff --git a/_posts/2017-09-19-iterative-security-devops-days-boston-2017.html b/_posts/2017-09-19-iterative-security-devops-days-boston-2017.html deleted file mode 100644 index 91a9ab9..0000000 --- a/_posts/2017-09-19-iterative-security-devops-days-boston-2017.html +++ /dev/null @@ -1,71 +0,0 @@ ---- -layout: post -title: Iterative Security - DevOps Days Boston 2017 -date: 2017-09-19 07:02:40.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- DevOps Days Boston 2017 -tags: -- DevOps Days Boston 2017 -meta: - _edit_last: '1' - _thumbnail_id: '813' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2017/09/iterative-security-devops-days-boston-2017/" ---- -Tom McLaughlin presented on iterative security, incorporating security into DevOps cycles through early detection and prevention of vulnerabilities. His slide are here. This is a loose transcript of taking points from his talk at DevOps Days Boston 2017.
-Breaches in Practice vs. Theory
-Tom made the point that breaches often occur in areas that aren't covered by development or security teams because vulnerabilities escape due to a lack of objective and continuous risk assessment.
-Code still has passwords and tokens in it. Lots of assumed knowledge going from dev to prod. Account access and password policies, patching, is usually handled by someone else. Leads to "good luck, it's up to you" syndrome.
-There's also security paralysis. When we don't think we know how to do something, we just won't. And we're rewarded for accomplishing things. So long as disaster doesn't strike, we get by.
-Why Do We Suffer from Security Breaches?
-Mostly, we get distracted. 0-day exploits, crypto weaknesses, hash collisions. We get distracted by logos and discussion threads, but not patching the system. We get caught up in all of this stuff instead of actually doing what improves security.
-
Think about all the publicly exposed mongodb and Elasticsearch instances you've seen...being proactive isn't always hard, but is rarely incentivized well.
-We don't do a good job explaining how to get from where you are to where you should be. We also don't always practice critical thinking. What is you goal? What is your posture about security? Proactive, reactive?
-We also don't always have a wealth of layered instructional content. There's a lot of information at the extremes (101 and advanced tutorials), but most of us are in the middle.
-Solve the Problem Like You're At Work
-So then let's develop a threat model together, as an example. Let's start by being realistic. What kind of org and product matters? Align with your company on risk management policies and processes.
-Prioritize. Use DREAD (or STRIDE) for rating threats and modeling risk.
-Also take care of the easy stuff: USB sticks over man in the ceiling.
-"Do you still use a service after it's been breached? I leave that up to you."
-Decompose the system. Map out your architecture and understand the systems. Look at the perimeters, how are credentials proliferated? Understand your data pipeline, where is your really valuable data stored?
-
Take time to consider things like exposed net ports, unpached containers, weak secrets...there are tools for this. These tools can be found in later slide here.
-Putting a Response to Security Threats into Action
-Two words: impose constraints. To find which constrains work for you and start with a simple discovery process that includes:
--
-
- Time, how long to solve? Timebox solutions, defensible use of existing time. -
- Complexity, how hard is it? Ask deep questions, iterate over which help. -
- Risk, how risky is the problem and solution? -
Secrets management is a first start. Tom pretty much pwns this space and I encourage you to seriously check out his extensive work on the topic here.
-In terms of tactical actions you can take today, Tom mentioned these few, but of course there are more:
--
-
- At the code, start with at least something like git-crypt.
-Ask yourself, what should be thrown out before it goes anywhere else?
- - In configuration management scripts:
-Developing a master re-key strategy is a great exercise to flesh this out.
- - Storage...a tool like sneaker for S3
-Really makes you ask questions...who/how are buckets managed.
-
Summary
-We need to be better at security, continuous or otherwise. We need to act. There are simple things you can do, but they need to be aligned to your team/organization risk strategy. And make it easy for others to do the right thing, so that it's far more likely to happen without imposing huge effort cost.
-Tom's a great speaker, engaging and fun to listen to. He is also a huge community contributor and even runs a distributed DevRel (developer relations) slack group. Tom is currently working on the CloudZero team.
-More reading:
--
-
- Tom's pantheon of posts at ThreatStack -
- Three Pillars of Continuous Security Improvement -
- Which Threat Risk Model is Right for Your Organization? -
[wonderplugin_slider id="1"]
diff --git a/_posts/2017-09-19-stop-using-the-staging-server-devops-days-boston.html b/_posts/2017-09-19-stop-using-the-staging-server-devops-days-boston.html deleted file mode 100644 index bc89af3..0000000 --- a/_posts/2017-09-19-stop-using-the-staging-server-devops-days-boston.html +++ /dev/null @@ -1,77 +0,0 @@ ---- -layout: post -title: Stop Using the 'Staging' Server - DevOps Days Boston -date: 2017-09-19 08:09:51.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- DevOps Days Boston 2017 -tags: -- DevOps Days Boston 2017 -meta: - _edit_last: '1' - _oembed_58580edecc96179de53c1702a687693f: "{{unknown}}" - _thumbnail_id: '807' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2017/09/stop-using-the-staging-server-devops-days-boston/" ---- -Chloe Condon presented on how containers and IaC (infrastructure as code) can help us skip over the 'staging server' part of traditional deployment strategies. This article is a loose transcript of taking points from her talk at DevOps Days Boston 2017.
-What's Wrong a Staging Environment?
-Feedback from a traditional staging environment is too slow. The only thing the reviewer knows is if unit tests passed, the rest of the tests are run after that. "Staging" is usually reserved for integration, functional, UI, and performance testing (i.e. complete feedback). Too little, too late.
-We're all too familiar with the question "who broke staging?". The fragility and centrality of this staging model creates bottlenecks. Also, the very first time something is brought into pipeline usually happens in staging and that's when 'broken' occurs.
-There's lots of "friction" between environments. Dev/test/staging are often not equivalent and are configured differently, causing deployment between environments to be a hastle. Flows across these environments are time-consuming (environment variables and files missing).
-Code changes are being tested more extensively in staging, which means there's little room for timely feedback.
-
Ephemeral Environments
-The great thing is now, we have containers. We can run every build, package it in a container, then run tests on it in the same pipeline. Microservices are well-suited for this type of model, but also distributed stacks (like a web app, database, and supporting APIs) benefit from this model too.
-Additionally, most stages of testing can be containerized. Leaving performance and scalability off for a moment, that enables us to run integration, functional, and security testing as part of a complete containerized package.
-The problem still remains: we have the rule that staging has to be as close to prod as possible. This might serve some of those tests (like performance and security), but is largely dis-optimal for unit, integration, and functional tests. Performance tests could also be run earlier to provide us a better heads-up about degradations that creep in over time. In practice, late-stage environments don't match reality and this causes friction..
-So let's reconsider the premise that all of our non-unit testing has to be run in a shared environment that bottlenecks us. This helps us shift feedback to the left. (Chloe says to insert Beyonce clip here.)
-
Containers = Consistency & Composition & Completeness
-So now the container we're handing off is much more complete: it includes a more complete set of self-testing capabilities that we can ask our pipeline to run for us.
-You can hand off containers to your customers (usually internal but maybe even external) and with composition, you have confidence that the bits they're running are the same as what you tested and what you want them to have.
-
Infrastructure as Code
-Team should define what code is part of the process. When people are able to spin things up automatically on their own, this streamlines an important part of their process. Visualizations help a lot, which is why CodeFresh and other platforms have visual controls over the package and deploy process.
-Infrastructure-as-Code (IaC) includes Dockerfiles, but also deployment scripts. If it's code, treat it like it's important because otherwise it's outside the flow of delivery.
-Paul's take: IaC also includes a whole bunch of other stuff too. For example:
--
-
- Composition scripts (like Docker compose, Kubernetes scripts) -
- Secrets management configuration -
- Network configuration -
- Database configuration (might include data) -
- Tests and test data -
- Feature flag configuration -
- Monitoring configuration & scripts -
Implementing IaC requires a few things:
--
-
- Your team agrees and has an in-depth knowledge of how to push healthy code artifacts into the pipeline. No one is an island, others' contributions need to be readable and easily debuggable. -
- A resilient process (i.e. pipeline) including dynamic build/package/test semantics enables contributors to focus on the 'push' and feedback rather than the semantics. -
- Information radiators along the process must cater feedback as granularly as possible: individual contributor first, then channel, then team. ChatOps bots give you immediate feedback about breakage as soon as it occurs. -
A complete IaC artifact list will require collaboration between multiple contributors, which facilitates communication. Just make sure that empathy and positive reinforcement is part of your management strategy.
-Questions from the Audience:
-Q: "How do you describe the state of the code in PRs?"
-Chloe: "Badges in the repo, some conventions, success flags on Codefresh."
-Q: "How often do people actually use this for pre-stage vs. just going to prod?"
-Chloe: "For lots of people, they maintain separate branches for multiple environments. Then you can introduce new versions dynamically."
-Q: "In more complex systems, is there a composition management layer?"
-Chloe: "This is the beauty of the compose files. When you treat them like code, this makes management a lot easier."
-More reading:
--
-
- Chloe on the CodeFresh Blog -
- Infrastructure as Code: a Reason to Smile -
- A Maturity Model for Infrastructure as Code -
[wonderplugin_slider id="1"]
diff --git a/_posts/2017-09-21-how-to-be-a-good-devops-vendor.html b/_posts/2017-09-21-how-to-be-a-good-devops-vendor.html deleted file mode 100644 index 31c0dc4..0000000 --- a/_posts/2017-09-21-how-to-be-a-good-devops-vendor.html +++ /dev/null @@ -1,111 +0,0 @@ ---- -layout: post -title: How to Be a Good DevOps Vendor -date: 2017-09-21 13:55:32.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- DevOps -tags: -- DevOps Days Boston 2017 -meta: - _edit_last: '1' - _thumbnail_id: '857' - _oembed_2efb77169ab4876e2f8ccf227c9591e9: - _oembed_time_2efb77169ab4876e2f8ccf227c9591e9: '1506608883' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2017/09/how-to-be-a-good-devops-vendor/" ---- -This article is intended for everyone involved in buying or selling tech, not just tooling vendors. The goal is to paint a picture of what an efficient supply and acquisition process in DevOps looks like. Most of this article will be phrased from a us (acquirer) to you (supplier) perspective, but out of admiration for all.
-Developers, Site-Reliability Engineers, Testers, Managers...please comment and add to this conversation because we all win when we all win.
-MP3: https://soundcloud.com/paulsbruce/how-to-be-a-good-devops-vendor
-[embed]https://soundcloud.com/paulsbruce/how-to-be-a-good-devops-vendor[/embed]
-I'll frame my suggestions across a simplified four stage customer journey:
- -Note: As you read the following statements, it may seem that I bounce around talking to various group in a seemingly random fashion. This is actually a result of looking at an organization through the customer lens across their journey. As organizations re-align to focus on delivering real value to customers, our paradigms for how we talk about "teams" also change to include everyone with a customer touch point, not just engineering teams.
-1. Make It Easy for Me to Try Your Thing Out
-(Product / Sales)
-Make the trial process as frictionless as possible. This doesn't mean hands off, but rather a progressive approach that gives each of us the value we need iteratively to get to the next step.
(Sales / Marketing)
-If you want to know what we're doing, do your own research and come prepared to listen to us about our immediate challenge. Know how that maps to your tool, or find someone who does fast. If you don't feel like you know enough to do this, roll up your sleeves and engage your colleagues. Lunch-n-learn with product/sales/marketing really help to make you more effective.
(Sales)
-I know you want to qualify us as an opportunity for your sales pipeline, but we have a few checkboxes in our head before we're interested in helping you with your sales goals. Don't ask me to 'go steady' (i.e. regular emails or phone calls) before we've had our first date (i.e. i've validated that your solution meets basic requirements).
(Product / Marketing)
-Your "download" process should really happen from a command line, not from a 6-step website download process (that's so 90s) and don't bother us with license keys. Handle the activation process for us. Just let us get in to code (or whatever) and fumble around a little first...because we're likely engineers and we like to take things apart to understand them. So long as your process isn't kludgy, we'll get to a point where we have some really relevant questions.
(Marketing / Sales)
-And we'll have plenty of questions. Make it absurdly easy reach out to you. Don't be afraid if you can't answer them, and don't try to preach value if we're simply looking for a technical answer. Build relationships internally so you can get a technical question answered quickly. Social and community aren't just marketing outbound channels, they're inbound too. We'll use them if we see them and when we need them.
(Marketing / Community / Relations)
-Usage of social channels vary per person and role, so have your ears open on many of them: Github, Stack Overflow, Twitter, (not Facebook pls), LinkedIn, your own community site...make sure your marketing+sales funnel is optimized to accept me in the 'right' way (i.e. don't put me in a marketing list).
-Don't use bots. Just don't. Be people, like me.
(Sales / BizDev)
-As I reach out, ask me about me. If I'm a dev, ask what I'm building. If I'm a release engineer, ask how you can help support your team. If I'm a manager, ask me how I can help your team deliver what they need to deliver faster. Have a 10-second pitch, but start the conversation right in order to earn trust so you can ask your questions.
-
2. Help Me Buy What I Need Without Lock-in
-(Sales / Customer Success)
-Even after we're prepared to sign a check, we're still dating. Tools that provide real value will spread and grow in usage over time. Let us buy what we need, do a PoC (which we will likely need some initial help with), then check in with us occasionally (customer success) to keep the account on the right train tracks.
(Sales / Marketing)
-Help us make the case for your tool. Have informational materials, case studies, competitive sheets, and cost/value break downs that we may need to justify an expenditure that exceeds our discretionary budget constraints. Help us align our case depending on whether it will be coming out of a CapEx or OpEx line. Help us make it's value visible and promote what an awesome job we did to pick the right solution for everyone it benefits. Don't wait for someone to hand you what you need, try things and share your results.
(Product)
-Pick a pricing model that meets both your goals and mine. Yes, that's complicated. That's why it's a job for the Product Team. As professional facilitators and business drivers, seek input from everyone: sales, marketing, customers!!!, partners, and friends of the family (i.e. trusted advisors, brand advocates, professional services). Don't be greedy; be realistic. Have backup plans on the ready, and communicate pricing changes proactively.
(Sales)
-Depending on your pricing model, really help us pick the right one for us, not the best one for you. Though this sounds counter-intuitive to your bottom line, doing this well will increase our trust in you. When I trust you, not only will I likely come back to you for more in the future, we'll also excitedly share this with colleagues and our networks. Some of the best public champions for a technology are those that use it and trust the team behind it.
3. Integrate Easily Into My Landscape
-(Product)
-Let us see you as code. If your solution is so proprietary that we can't see underlying code (like layouts, test structure, project file format), re-think your approach because if it's not code, it probably won't fit easily into our delivery pipeline. Everything is code now...the product, the infrastructure configuration, the test artifacts, the deployment semantics, the monitoring and alerting...if you're not in there, forget it.
(Product)
-Integrate with others. If you don't integrate into our ecosystem (i.e. plugins to other related parts of our lifecycle), you're considered a silo and we hate silos. Workflows have to cross platform boundaries in our world. We already bought other solutions. Don't be an island, be a launchpad. Be an information radiator.
(Product / Sales / Marketing)
-Actually show how your product works in our context...which means you need to understand how people should/do use your product. Don't just rely on screenshots and product-focused demos. Demonstrate how your JIRA integration works, or how your tool is used in a continuous integration flow in Jenkins or Circle CI, or how your metrics are fed into Google Analytics or Datadog or whatever dashboarding or analytics engine I use. The point is (as my new friend Lauri says it)..."show me, don't tell me".
(Sales / Marketing)
-This goes for your decks, your videos, your articles, your product pages, your demos, your booth conversations, and even your pitch. One of the best technical pitches I ever saw wasn't a pitch at all...it was a technical demo from the creator of Swagger, Tony Tam at APIstrat Austin 2015. He just showed how SwaggerHub worked, and everyone was like 'oh, okay, sign me up'.
-Truth be told, I only attended to see what trouble I could cause. Turns out he showed a tool called Swagger-Inflector and I was captivated.
-- Darrel Miller on Bizcoder
(Sales / Product)
-If you can't understand something that the product team is saying, challenge them on it and ask them for help to understand how and when to sell the thing. Product, sales enablement is part of your portfolio, and though someone else might execute it, it's your job to make sure your idea translates into an effective sales context (overlap/collaborate with product marketing a lot).
(Product / Customer Support)
-As part of on-boarding, have the best documentation on the planet. This includes technical documentation (typically delivered as part of the development lifecycle) that you regularly test to make sure is accurate. Also provide how-to articles that are down to earth. Show me the 'happy path' so I can use it as a reference to know where I've gone wrong on my integration.
(Product / Developers / Customer Support)
-Also provide validation artifacts, like tools or tests that make sure I've integrated your product into my landscape correctly. Don't solely rely on professional services to do this unless most other customers have told you this is necessary, which indicates you need to make it easier anyway.
(Customer Support / Customer Success / Community / Relations)
-If I get stuck, as me why and how I'm integrating your thing into my stuff to get some broader context on my ultimate goal. Then we can row in that direction together. Since I know you can't commit unlimited resources to helping customers, build a community that helps each other and reward contributors when they help each other. A customer gift basket or Amazon gift card to the top external community facilitators goes a long way to gaming yourself into a second-level support system to handle occasional support overflows.
4. Improve What You Do to Help Me With What I Do
-(Product / Development / Customer Support)
-Fix things that are flat out broken. If you can't now, be transparent and diplomatic about how your process works, what we can do as a work-around in the mean time, and receive our frustration well. If we want to contribute our own solution or patch, show gratitude not just acknowledgement, otherwise we won't go the extra mile again. And when we contribute, we are your champions.
(Product)
-Talk to us regularly about what would work better for us, how we're evolving our process, and how your thing would need to change to be more valuable in our ever-evolving landscape. Don't promise anything, but also don't hide ideas. Selectively share items from your roadmap and ask for our candid opinion. Maybe even hold regional user groups or ask us to come speak to your internal teams as outside feedback from my point of view as a customer.
(Product)
-Get out to the conferences, be in front of people and listen to their reactions. Do something relevant yourself and don't be just another product-headed megalomaniac. Be part of the community, don't just expect to use them when you want to say something. Host things (maybe costs money), be a volunteer occasionally, and definitely make people feel heard.
(Everyone)
-Be careful that your people-to-people engagements don't suffer from technical impedance mismatch. Sales and marketing can be at booths, but should have a direct line to someone who can answer really technical questions as they arise. We engineers can smell marketing/sales from a mile away (usually because they smell showered and professional). But it's important to have our questions answered and to feel friendly. This is what's great about having your Developer Relations people there...we can nerd out and hit it off great. I come away with next steps that you (marketing / sales) can follow-up on. And make sure you have a trial I can start in on immediately. Use every conversation (and conference) as a learning opportunity.
(Product)
-Build the shit out of your partner ecosystem so it's easier for me to get up and running with integrations. Think hard before you put your new shiny innovative feature in front of a practical thing like a technical integration I and many others have been asking for.
(Development / Community / Marketing / Relations)
-If there is documentation with code in it and you need API keys or something, inject them in to the source code for me when I'm logged in to your site (like SauceLabs Appium tutorials). I will probably copy and paste, so be very careful about the code you put out there because I will judge you for it when it doesn't work.
(Marketing / Product)
-When you do push new features, make sure that you communicate to me about things I am sure to care about. This means you'll have to keep track of what I indicate I care about (via tracking my views on articles, white paper downloads, sales conversations, support issues, and OPT-IN newsletter topics). I'm okay with this if it really benefits me, but if I get blasted one too many times, I'll disengage/unsubscribe entirely.
Summary: Help Me Get to the Right Place Faster...Always
-None of us have enough time for all the things. If you want to become a new thing on my plate, help me see how you can take some things off of my plate first (i.e. gain time back). Be quick to the point, courteous, and invested in my success. Minimize transaction (time) cost in every engagement.
-(Sales, et al: "Let's Get Real or Let's Not Play" is a great read on how to do this.)
-At often as appropriate, ask me what's on my horizon and how best we can get there together. Even if I'm heads-down in code, I'll remember that you were well-intentioned and won't write you off for good.
-NEXT STEPS: share your opinions, thoughts, suggestions in the comments section below! Or ping me up on Twitter@paulsbruce or LinkedIn.
-[wonderplugin_slider id="1"]
-More reading:
--
-
- RETROs of DevOps Days Boston 2017 -
- DevOps is Organizational, Operational, and Orthogonal -
- Being a Great Technical Evangelist Means Being in the Right Room -
In a conversation today with Ken Mugrage (organizer of DevOps Days Seattle), the scope of the term 'DevOps' came up enough to purposely double-click into it.
-'DevOps' Is (and Should Be) Limited In Scope
-Ken's view is that the primary context for DevOps is in terms of culture, as opposed to processes, practices, or tools. To me, that's fine, but there's so much not accounted for that I feel I have to generalize a bit to get to where I'm comfortable parsing the hydra of topics in the space.
-Like M-theory which attempts to draw relationships in how fundamental particles interact with each other, I think that DevOps is just a single view of a particular facet of the technology management gem.
-DevOps is an implementation of a more general theory, a 'next' mindset over managing the hydra. DevOps addresses how developers and operations can more cohesively function together. Injecting all-the-things is counter to the scope of DevOps.
-Zen-in: A New Management Theory for Everyone
-Zen-in (ぜんいん[全員]) is a Japanese term that means 'everyone in the group'. It infers a boundary, but challenges you to think of who is inside that boundary. Is it you? Is it not them? Why not? Who decides? Why?
-By 'management' theory, I don't mean another 'the silo of management'. I literally mean the need to manage complexity, personal, technological, and organizational. Abstracting up a bit, the general principals of this theory are:
--
-
- Convergence (groups come together to accomplish a necessarily shared goal) -
- Inclusion (all parties have a voice, acceptance of constraints) -
- Focus (alignment and optimization of goal, strategies, and tactics) -
- Improvement (learning loops, resultant actions, measurement, skills, acceleration, workforce refactoring, effective recruiting) -
- Actualization (self-management, cultural equilibrium, personal fulfillment) -
I'll be writing more on this moving forward as I explore each of these concepts, but for now I think I've found a basic framework that covers a lot of territory.
-I Need Your Help to Evolve This Conversation
-True to Zen-in, if you're reading this, you're already in the 'group'. Your opinions, questions, and perspectives are necessary to iterate over how these concepts fit together.
-Share thoughts in the comments section below! Or ping me up on Twitter@paulsbruce or LinkedIn.
-diff --git a/_posts/2017-09-26-recap-of-devops-days-boston-2017-with-corey-quinn.html b/_posts/2017-09-26-recap-of-devops-days-boston-2017-with-corey-quinn.html deleted file mode 100644 index 43d6c2e..0000000 --- a/_posts/2017-09-26-recap-of-devops-days-boston-2017-with-corey-quinn.html +++ /dev/null @@ -1,41 +0,0 @@ ---- -layout: post -title: Recap of DevOps Days Boston 2017 with Corey Quinn -date: 2017-09-26 15:18:22.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- DevOps -- DevOps Days Boston 2017 -tags: -- informal interface -meta: - _edit_last: '1' - _oembed_0b65e1c292f5e24dfa0c98ff8c0a3ed5: - _oembed_time_0b65e1c292f5e24dfa0c98ff8c0a3ed5: '1509540074' - _thumbnail_id: '890' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2017/09/recap-of-devops-days-boston-2017-with-corey-quinn/" ---- -
This weekend, I had the chance to have a 'distributed beer' with Corey Quinn of Last Week in AWS to chat about the DevOps Days Boston 2017 conference last week. We provide a few takeaways each in about 5 minutes.
-You can watch it on Youtube and listen on Soundcloud.
-[embed]https://youtu.be/LvJffvi5J0E[/embed]
-My Recaps of Day 1:
-[wonderplugin_slider id="1"]
-.
-Links from this chat:
--
-
- Subscribe to Last Week in AWS -
- Recap of 'Stop Using the Staging Server' by Chloe Condon -
- Slides from 'The Curse of Common Knowledge' by Yulan Lin -
diff --git a/_posts/2017-09-28-folding-open-source-into-enterprise-devops.html b/_posts/2017-09-28-folding-open-source-into-enterprise-devops.html deleted file mode 100644 index 49aa8fa..0000000 --- a/_posts/2017-09-28-folding-open-source-into-enterprise-devops.html +++ /dev/null @@ -1,67 +0,0 @@ ---- -layout: post -title: Folding Open Source into Enterprise DevOps -date: 2017-09-28 17:02:50.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- DevOps -- open source -tags: -- DevOps -meta: - _edit_last: '1' - _thumbnail_id: '937' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2017/09/folding-open-source-into-enterprise-devops/" ---- -
Open source software (OSS) is a foundational part of the modern software delivery lifecycle. Enterprise teams with DevOps aspirations face unique challenges in compliance, security, reliability, and sustainability of OSS components. Organizations-in-transformation must have a complete picture of risk when integrating open source components.
-This article explores how to continuously factor in community and ecosystem health into OSS risk analysis strategy.
-The Acquisition Process for Open Source Software
-Developers need to build on the successes and contributions of others. Having the flexibility to integrate new open source components and new versions of existing dependencies enables teams to go fast, but external code must be checked and validated before becoming part of the trusted stack.
-Including someone else's software is an important moment of engagement. Enterprises typically wrap a formal 'acquisition' process around this process, where the 'supplier' (the entity who provides the software/service) and the 'acquirer' (the entity who wants to integrate the software/service) contractualize.
-Though there are already commercial approaches to introducing software packages safely by companies like Sonatype, Black Duck, and others, my question extends beyond the tools conversation to encompass the longer-term picture of identifying and managing risk in software delivery.
--Enterprises care deeply about risk. Without addressing this concern, engineering teams will never actualize the benefits of DevOps.
This is a tangible application of the need for DevOps to not only apply at an individual team level, but in the broader organization as well. It takes alignment between a team who needs software and teams providing compliance and legal services to all do so in an expedient manner that matches the clock speed of software delivery.
-Communities Empower Enterprises to Address this Gap
-Today in a Global Open Source Governance Group Chat, I asked the question:
-"What are some methods for determining how significant a supplier/vendor OSS and community contributions are, relative to acquirer confidence?"
-This question stems from my involvement with the IEEE 2675 working group, particularly because I see:
--
-
- Prolific use of OSS use in DevOps and in enterprise contexts -
- Reluctance and concern (rightly so) around integration of OSS in enterprise software development and operation in production -
- The convergence of compliance and automation considerations -
- How important transparency and collaboration is to the health of OSS, but also to the supply and acquisition processes in a DevOps lifecycle -
The group, expertly facilitated by Lauri Apple, also included key insights from Paul Burt and Jamie Allen. A log of the conversation can be found on Twitter.
-As open source projects (like Swagger/OADF for instance) become increasingly important to enterprise software delivery, health and ecosystem tracking also becomes equally important to any new components being introduced.
-My point-of-view is that organizations should prepare a checklist for software teams to construct a complete picture of risk introduced by OSS (not to mention proprietary) components. This checklist must include not only static analysis metrics but support, engagement, funding, and contribution considerations.
-Measuring OSS Project + Community Health
-The group had many suggestions that I wouldn't have otherwise thought about, another reason for more people getting involved in dialogs like this.
-There are already providers of aggregate information on open source community health and contribution metrics such as CHAOSS, a Linux Foundation project, and Bitergia. This data can be integrated easily into dependency management scripts in Groovy, npm, Ant, Maven, etc. and at the very least written in to a delivery pipeline as part of pre-build validation (BVT is too late).
-And there is honest, hard-hitting research on open source software...which you should take the time to read....from Nadia Eghbal under the Ford Foundation in a report called 'Roads and Bridges: The Unseen Labor Behind Our Digital Infrastructure'. If you don't have time to read it, buy some text-to-speech software and listen to it when you're in transit.
-The group also identified some key characteristics of OSS community health not necessarily tracked by established services, such as:
--
-
- Same day response on reported issues, even if it's simply acknowledgement -
- PRs under the "magic number" of 400 lines of code...tends to be the limit for # of bugs and useful feedback -
- Outage response, sandbox availability -
- Distribution of component versions across multiple central repositories -
More to Come...From YOU
-As I integrate both my own learnings and other voices from the community into the larger Enterprise DevOps conversation, the one thing that will be missed is YOUR THOUGHTS, whether your in a large organization or simply in a small team.
-Please share your thoughts in the comments section below! Or ping me up on Twitter@paulsbruce or LinkedIn.
-More reading:
--
-
- How to Be a Good DevOps Vendor -
- DevOps is Organizational, Operational, and Orthogonal -
- Enterprise Wild West - DevOps Days Boston 2017 -
- Swagger => OADF: Insider View -
This week, I had the opportunity to continue a conversation started weeks ago with David Fredricks, organizer of DevOps Days Boston.
-You can watch it on YouTube or listen via Soundcloud. Transcript below.
-https://youtu.be/SUKOShqBuPE
-The Incredible Impact of Open Spaces
-Paul: All right so welcome everyone my name is Paul Bruce and once again I'm back here with a member of the DevOps community, Dave Fredericks. Now Dave you organize the DevOps Boston event is that correct?
-Dave: Yeah that's correct. I've participated as a volunteer organizer for the last three years, involved for the last four.
-Paul: Excellent...and I got a chance to meet you beforehand, I think at one of the DevOps Days Boston meetups, but then also we got to chat at the event and it was a really good event. I think a number of different things were really just cohesed really well, particularly from my point of view, the collaborative open spaces. Can you tell us a little bit about what that's like how that got into the conference schedule?
-Dave: Yes, certainly. So open spaces is a really interesting kind of platform that's unique to DevOps Days. Basically how it works is everybody comes up with topics during the event after listening to some of the keynotes. Some discussions that are interesting to individuals, a lot of times you want to add on your personal perspective into, not only offering new ideas or maybe even some suggestions, but asking specific questions.
-A way of being able to do that is by getting everyone together at the event over common topics. You basically vote on different topics that are of interest to you and they can actually go anywhere from cultural to personal to technology as a whole.
-The idea is, there's a few rules, it's basically:
--
-
- what's being said is what needs to be said -
- who's there is the people that need to be there -
- when it starts and when it ends is the time it starts and ends -
Those are the only kind of guidelines that we go by and the idea is to get people who usually wouldn't be open to public speaking to be able to have a chance and an opportunity to either share some ideas, ask questions specifically and directly to different individuals and to have an open forum.
--The real values that come out of it are real specific dialogue, the biggest thing is new introductions and relationships that are created.
The hope is that throughout the year after the event, a DevOps stage event is for you to be able to get contact information of individuals who are in the same space, at the same stage as yourself to have an outside outlet to be able to bounce ideas off of through the year as you start to face some of the challenges as you as an engineer try to solve problems.
-Paul: Yeah that was one of those things that really clicked for me, being part of a number of those open spaces, I saw exactly what you said which was people were far more likely to comment and to share and ask questions. And in a larger audience and I think the other element of that is the fact that not only do they get the share but they get instant feedback.
-And this is one of those core tenants I think of DevOps, in my mind, is this concept of continuous learning. But you don't learn unless you know what's going on and you don't know what's going on unless you [as an organization] radiate information which is typically facilitated by feedback loops. So whether we're talking about technology feedback loops or real people feedback loops, I think that's really helpful.
-So can I back up for a second and ask you a slightly broader question about DevOps: in your mind how would you define DevOps?
-What Is DevOps, Really?
-Dave: Great question. You know these this is one that in our community we talk a lot about, especially for folks who are outside of quote-unquote "DevOps thought process", knowing that it's something that's taking off as a force in the software world.
-One of the things we do is to talk about how do we define DevOps. The biggest thing for me is DevOps means different things to different people and it's all about context and perspective, where you come from and where you've been and what challenges you're trying to solve. So when I meet somebody new who's in this space and they're starting to kind of either chant or evangelize to me without first getting a baseline perspective as to where I'm at and what I was doing and what I'm trying to solve, immediately has me question, "okay, are you trying to push your ideals down on me?",
--This is what DevOps means to me: getting folks to work together in an efficient collaborative manner to solve a common goal, period.
It has nothing to do with tools. It has nothing to do with process. It has nothing to do with frameworks. It's all about getting people together, teaching context, having empathy, understanding what somebody's doing, why they need to do it, and what what they've been doing in the past. You share your ways of doing it and then together when you have a sense of "okay, I know why this person has to do things, I know the reason why they're thinking this way", you can efficiently solve problems and for me that's that's what DevOps is to the core, right there.
-Paul: So one one thing I heard from that is it starts with people, right? It doesn't start with tools, it doesn't start with how you've been doing it; it starts with people and really understanding the context and the perspective that they bring to the table. Is that right?
-Dave: Yeah, Paul, you you nailed it right there. It starts, it continues, and it ends with people. Ultimately I take the concepts and the core principles of DevOps, and you can apply that to any industry, any product, any delivery, any manufacturing, and it really is bringing people together to work more efficiently to solve a common problem.
-What Is DevOps Not?
-Paul: And so actually people are doing that, you'll hear the prevalence of these amalgam terms like DevSecOps, DevTestQAOps. And I kind of take issue with that in the sense that I understand how important terminology and clear labels for things. As a practitioner and engineer, as soon as somebody starts to blow out a term to mean "all the things", my red flags get raised up instantly.
-That doesn't mean that [DevOps] doesn't include other people, but can you tell us a little bit about how important the scope is of DevOps to you? And just kind of following that up with some context, I was able to speak to Ken Mugrage from the DevOps Days Seattle, and he was very clear about how if we blow it out into all the things, "DevOps" loses its value.
-And so I put this to you: why is a pantheistic term, if DevOps grows to that, why is that a problem?
-Dave: No, that's a great thought. I want to take this back a little bit to identify why are all these actions added on, how and why this is how [DevOps] is being branded in this way. This was a discussion that I have, especially with growing teams.
--One of the biggest things I talked about with organizations is, first and foremost, technically there is no DevOps engineers. So why label it that way?
There's No Such Thing as a "DevOps Engineer"
-When I started working with a lot more enterprises, I helped organizations transform their development to be much more modern so that they can have quicker release cycles and feedback. It's one of the things that used to frustrate me, was like "hey, we need five DevOps engineers!". That doesn't mean anything to me, you got to explain on a day to day basis, what is this person doing, and ultimately, why are you labeling these folks as DevOps engineers?
-And I I had some interesting feedback which came from the product marketing side. They were like, "Dave, we're in the enterprise. We're used to big long deploys of software in order to get it to our customers, and a lot of times we don't know if our customers are even getting any value out of what we're producing. When we're releasing every year and waiting for six months to get the actual feedback from our customers, it doesn't make any sense."
-So you see this large swath of folks trying to get into this space to build software quicker to have faster feedback to be able to add more value to end users.
--These individuals don't really understand this whole open source community, they don't understand how the strength of the community is really the value.
"So we don't know how to really market. We don't know how to communicate to the group in a way for us to be able to blanket it all together. So we just scoped it into this thing and we call it #DevOps and everything gets that kind of label to it."
-From my experience what I'm starting to see is a lot more of these organizations who are specific to security, to testing, in a way of being able to catch and grasp that member of the audience, it's "let's throw it in, Dev and Sec Ops, Dev Quality Ops. What starts to happen in my mind and what I'm what I'm worried about is that people start to lose the real purpose.
-Paul: So basically the exact same thing that happened to Agile. Everybody forgot to have agility as one of the core tenants that people check in on, on a regular basis such that they internalize that, and that is where their activities and their tools flow from, right?
-Dave: Yes exactly. If you start to get too focused on the terminologies and the labeling of things and forget the context as to why you're practicing it, ultimately the further down stream you get and the more generations that start to get folded into the process, they'll start to lose the actual scope, "hey we're trying to get people to to work together in a more collaborative manner to be efficient and to be able to deliver quickly."
-How to Be a Good DevOps (Citizen, Vendor, Employer)
-Paul: Yeah, one thing that I did recently was put out an article (and thank you you, you had shared it to a number of people and I think that's half the reason why I got some attention). It was essentially how to be a good DevOps vendor. It took the approach of looking at it from the customers perspective. The implementation of that was over a simplified customer journey and then chronologically through that journey, I went through and basically made statements from an outsider's perspective onto different groups whether it be product, marketing, sales.
-Back to your perspective, I get that it has to fundamentally start with people because people are what build teams and teams are what build software and software is what affects us. But the team affects us and individuals affect us, and so it does make sense to keep that as a core of value, to consider personal responsibility and also the responsibility of the team to have these cultural aspects present.
-But unfortunately I think what happens is that we do need tools and you know, conferences are notorious for needing some kind of funding and becoming self-funding is really hard, and so out comes sponsor packages and I mean it's an ecosystem. All software is eventually, in most people's minds, going to make money and so this is where I was coming from, understanding that there is no such thing as a DevOps vendor or a DevOps tool or a DevOps job/position. Yet the fact is that when you're closely aligned with the thinking of another person and "DevOps" is the term they're using, it's easy for these vendors to kind of bring that in and pull that into their messaging.
-So I guess my my point of view on that is that we are gonna have to deal with that but it's kind of a constant battle against the pantheism of trying to "all the things" a term [DevOps] but in the meantime we also do have to represent those tenants to more than just the developers and operations. If you really want to sell to developers and operations or teams that are looking, or they have internalized DevOps, they're going to be looking at the world from this interesting perspective. And they'll be looking across the tool chain to figure out who sounds like they're blowing smoke up [you know where].
--If a tool vendor or a service provider does not understand the core of DevOps, then their messaging, their selling process, their product ideation...it's all not going to jive with the real market.
After after a recent Boston DevOps meetup we dove into this for what like an hour and a half, and just really talked about how do we actually do this. My concern is that when we start to move this into the enterprise (and by the way, the good principles of DevOps should be moveable to the enterprise, right? If they work, they work, and it's a matter of fitting to context) that I think, while the core of it is culture, we can't just live in this sort of kumbaya world.
--We really have to figure out how to scale DevOps principals up and out into the enterprise setting so that, by the way, these good principles have a positive impact on things like automated insurance, things like machine learning in terms of healthcare, defense and government settings.
So I'm working on that on the side but in the meantime, what do you think about scaling to the enterprise? What does that even mean for DevOps?
-How DevOps Is Re-writing Management Decisions
-Dave: Yeah, that's a great point. It's an interesting challenge. There's a lot of organizations who are facing it. Right now, I'm dealing with situations where we're starting to see a lot of enterprise buy instead of trying to build it themselves. One thing they have is capital and resources. So the idea is, "if we don't know or we can't make it, it's the bye versus build, like why go out and try to do what people already are being really successful in doing in something that we don't understand too well? Let's just go ahead and absorb some of these startups..."
-Paul: Do you mean actually purchasing startups in order to just fill that technical gap in an organization? So I don't want to name names, but I'm thinking of a very large enterprise that just recently bought up one of the most well-known API monitoring services out there, and people are freaking out like "oh gosh, what's going to happen, are they going to de-culture this awesome group of guys and gals?"
-Dave: I'm dealing with the same thing within an organization, a large security company buying a smaller more nimble security product with a lot of open source options. They're putting out there trying to create groundswell to get this tool for free into the hands of engineers, let them play with it so they can understand how it works and create some kind of a swell within the engineering teams and then we'll come up to the top start talking to the executives about, "Hey, what challenges are you facing in this broad space?", where you're trying to protect not only year your customers information but also information about your company.
-As they start to have that type of dialogue, all of a sudden the executives within the organization starts to look down, talking to their engineering group and saying "hey, what do you know, what have you played with, what do you think is interesting, how do you think we should be solving this problem?" You've already created that initial lift of inertia in engineering, then they say hey let's go with this product...we already know how it works, we've been you tooling around with it. Win-win, right?
-So this is a completely different way of thinking of how enterprises used to be selling products into their customers. It was always a top-down approach...let's talk to the executives who have the purchase power, float it down and then they'll disseminate that information in the way that we roll it out into the engineering team. That's how you could do it in the old-school way. Now in today's new world, a lot of tools are available for you to play with for free and when enterprise organizations start to try to come into this space, they're really kind of blindsided by this whole new content creation process.
-Selling Into DevOps Takes Understanding DevOps
-What I'm starting to see is they're at least now recognizing we do not know how to sell to this to this community of this group. We know we really want to get into the space, we want to do it the right way, what do we do right and you know to your point with your article, I've shared your article with all of the enterprises that I've have been talking to me about this problem because I can't teach them about the thought process of open source.
-I mean, we can look back in the 60s, the MIT days, where the two groups kind of split off. A lot of us in the DevOps space already have the mentality of like "hey, you know we want to be able to share a lot of this stuff but we do want value for hard work we do". But for the most part there's different ways of doing it versus everything is being paid for with the enterprise mentality.
-What I'm starting to experience is there's a lot of organizations out there that are realizing it's exponential value once they start to get into this community and...
--the brand loyalty within the DevOps community is tremendous
...but the challenge that is in front of us right now is really the learnings piece and I'm thinking it's a leadership issue (this is my own personal view). It's enterprise leadership that needs to get out of the way and allow for new blood to come in to be able to understand the kind of movement. I've been doing a little, as much as I can to try to influence old leadership. It's a challenge and a lot of it has to do with success syndrome. You've been doing it in certain way for decades. It's a great case study that we're gonna be able to kind of sit back and watch in the next five years
-Calling All Researchers: Inclusion Means You Too!
-Paul: Yeah and you know, there's so much going on, no one person can do it alone. So without plugging any commercial products of any kind (that's not my motion) I have started something called the iterativeresearch.org, which is essentially a bunch of contributors to research. As they go along, it could be lightweight contributions, simply just pocketing articles and getting into a feed of people who pay attention, it's writers too, but the point is it's not on a brand that's connected to a pay-for services. And you know I would love, for this conversation to really start flowing in that direction because I think it takes many perspectives, right?
--The core of this is it's an inclusive conversation, not an exclusive one.
So understanding that you are a busy man and we're at the top of our time, are there one or two things that you want to give a shout-out to or any particular resources that people can go to, events, communities, open-source forums, anything like that?
-Get Out to a DevOps Tribe Near You!
-Dave: Yeah, you know, thank you so much for the opportunity first and foremost, we're gonna have to do it again! One of the things I really would highly recommend to folks who are interested in getting more involved, start to look at some local meetups that you have going on. There are some great folks within every community in whatever city, whatever small town, who are interested in sharing ideas and in thoughts in challenges. All you have to do is get out there and look. Go find your tribe! The biggest thing is don't sit back and wait and sit on your hands and expect for interest to come to you.
--The whole constant learner, the Kaizen mentality, be better tomorrow than you are today, be better today than you are yesterday. It lives and dies in DevOps and the way to do it is start to talk to folks who you're not used to talking to.
Don't be afraid get out there introduce yourself and have a good time. Life is learning.
-Paul: Cool. So that's David Frederick's everyone and thank you David for spending the time with me. Do you prefer going by David, Dave?
-Dave: Dave, David, either way.
-Paul: Dave/David, I've really enjoyed it was great being able to spend some time. We'll circle back. Thank you so much! Cheers!
--
More from DevOps Days Boston:
-[wonderplugin_slider id="1"]
--
diff --git a/_posts/2018-01-25-setting-up-your-own-selenium-grid-on-aws.html b/_posts/2018-01-25-setting-up-your-own-selenium-grid-on-aws.html deleted file mode 100644 index 5175f17..0000000 --- a/_posts/2018-01-25-setting-up-your-own-selenium-grid-on-aws.html +++ /dev/null @@ -1,86 +0,0 @@ ---- -layout: post -title: Setting Up Your Own Selenium Grid on AWS -date: 2018-01-25 13:56:54.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- automation -- Selenium -- testing -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '995' - dsq_thread_id: '6440200396' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2018/01/setting-up-your-own-selenium-grid-on-aws/" ---- -
This article describes how DevOps teams can quickly spin up a reliable, cost-effective Selenium grid for automated testing in minutes.
-What Is Selenium & Selenium Grid?
-If you've never heard of Selenium before, climb out from under that rock.
-Selenium is a test automation framework that lets you script interactions against your web apps and run them in popular web browsers. Selenium Grid is a management service that coordinates the use of many instances of these browsers. You can point your Selenium scripts to a Grid, specify which "capabilities" (OS version, browser version, etc.) it would like in an interactive session, then drive actions in that browser session to verify functional correctness of a target app or website.
-
This is what it typically looks like in Selenium code:
-
Pretty simple, really, for what's really going on under all the layers of code and HTTP protocol communications.
-Why Use a AWS for Your Selenium Grid
-There are plenty of services that host Selenium-compliant grids, such as Sauce Labs and Browser Stack, and I recommend you consider one of the many SaaS options if you don't want to maintain and upgrade a grid on your own. However, if your team has reasonable automation skills and the human resources to do so, the benefit of a DIY grid is complete control for a fraction of the cost associated with hosted services.
-Though I abhor the idea of paying hundreds of dollars per month for only a few parallel browser instances, a word of warning about DIY grids over SaaS alternatives:
--
-
- You don't get same-day support for the latest browsers and OS versions -
- You don't benefit from script-to-driver version independence -
- You don't get infinite scalability without some serious effort -
- You don't get 24/7 operational oversight unless you pay someone to do it -
If you (and your team) are okay with the above disclaimers and simply just want a quick, cheap way to spin up a grid, then the steps below suffice:
--
-
- Launch a Ubuntu linux distro to act as the Docker host.
-I used an t2.xlarge at $0.19/hr to get 4vCPU variable and 16GB memory. Also set SDA storage to 32gb.
- - Log in to your instance via SSH -
- Switch to root account to install Docker with 'sudo su' command -
- Install cAdvisor with this command:
-
-sudo docker run \ - --volume=/:/rootfs:ro \ - --volume=/var/run:/var/run:rw \ - --volume=/sys:/sys:ro \ - --volume=/var/lib/docker/:/var/lib/docker:ro \ - --volume=/dev/disk/:/dev/disk:ro \ - --publish=8080:8080 \ - --detach=true \ - --name=cadvisor \ - google/cadvisor:latest
- - Install Selenium Hub:
-
$ docker network create grid -$ docker run -d -p 4444:4444 --net grid --name selenium-hub selenium/hub:latest
-
- - Create containers for your various browsers using debug versions and unique external port numbers (typically 5901, 5902, etc.):
-
$ docker run -d -P -p <port4VNC>:5900 --net grid -e HUB_HOST=selenium-hub -v /dev/shm:/dev/shm selenium/node-chrome-debug:latest -$ docker run -d -P -p <port4VNC>:5900 --net grid -e HUB_HOST=selenium-hub -v /dev/shm:/dev/shm selenium/node-firefox-debug:latest
-
- - In the security group you created/assigned during EC2 instance setup, add the following inbound rules:
-7777 (cAdvisor)
-4444 (Selenium Hub)
-5900-6000 (VNC for various browsers)
- - Verify that your grid contains the browsers you spun up:
-http://[host-ip]:7777/
- - Connect to VNC remote services on instances using the ports assigned in the above docker run commands. On a Mac, I recommend VNC Viewer. -
- Reference your grid in a Selenium script, using port 4444: -
[picture of code]
-Now when you run your script(s), particularly in parallel, you should see nodes on your grid running your automated script.
-How Is a Selenium Grid Better Than Local Execution?
-Well, for one, it's not popping up multiple browsers on your workstation while you're trying to get other work done. In build and continuous integration scenarios, you need to run automated Selenium scripts on an environment that doesn't include the laptops your developers take home with them at night.
-You also want your test environment to be as versioned and reliable as possible, so having an on-demand grid that can be scripted as an AWS YAML descriptor and that meets your manual, on-demand, and CI testing requirements is a modern must.
-Why Did I Write This Tutorial?
-I'm a sometimes performance engineer. I needed a disposable Selenium Grid to show how to use a subset of browser sessions during a large load test to collect real-user experience metrics in NeoLoad. I didn't want to pay hundreds of dollars for a monthly subscription to something I could build in about 30mins.
-Why am I running a subset of real browsers as part of a load test? That, my friends, is the really interesting part that I discuss in a later post.
diff --git a/_posts/2018-02-11-devops-testing-strategy-for-dummies.html b/_posts/2018-02-11-devops-testing-strategy-for-dummies.html deleted file mode 100644 index 3702618..0000000 --- a/_posts/2018-02-11-devops-testing-strategy-for-dummies.html +++ /dev/null @@ -1,55 +0,0 @@ ---- -layout: post -title: DevOps Testing Strategy for Dummies -date: 2018-02-11 10:12:19.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- automation -- DevOps -- testing -tags: [] -meta: - _thumbnail_id: '1014' - _edit_last: '1' - _oembed_7d59fb83d3de7837fd3a9e372f753e94: "{{unknown}}" - dsq_thread_id: '6484785155' - _wp_old_slug: devops-testing-strategy -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2018/02/devops-testing-strategy-for-dummies/" ---- -Testing in a DevOps culture is very different from traditional QA scenarios. I talk to all kinds of teams, from Fortune 100 to startups, all on the journey to adapt and innovate. What happens to testers in this new world?
-This article is a bundle of content related to why and how software teams can align and improve their testing strategy. I addresses "right fit" to across org and process, cost center vs. value stream, and many other dynamics in testing culture.
-|
-
"No Testing Strategy, No DevOps"
"Testing Strategy in DevOps" |
-
-
- Greater Boston Selenium Users Group
Soundcloud: background on |
-
diff --git a/_posts/2018-02-24-the-nra-magazines-and-video-games.html b/_posts/2018-02-24-the-nra-magazines-and-video-games.html deleted file mode 100644 index fd046d2..0000000 --- a/_posts/2018-02-24-the-nra-magazines-and-video-games.html +++ /dev/null @@ -1,60 +0,0 @@ ---- -layout: post -title: The NRA, Magazines, and Video Games -date: 2018-02-24 18:36:13.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: [] -tags: [] -meta: - _edit_last: '1' - _oembed_67aab636628a71d0e4c4953586b39179: "{{unknown}}" - _thumbnail_id: '1032' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2018/02/the-nra-magazines-and-video-games/" ---- -
I think we forget that we have trained a generation of children to commit mass shootings. "Video Trainings" like Call of Duty and Battlefield should be declassified as "video games" because killing is not a game. Likewise, we should re-sensitize to how alluring the symbol of a gun is to adults who need to feel powerful.
-NRA and Magazines
-Yesterday, a neighborhood friend posted a photo of gun magazines in epic display at one of our local chain pharmacies. After mulling over what this meant for about this for 2mins, I responded:
-
Granted, this is a liberal-as-fuck move, using social media to push against a brand publicly to take ownership for businesses and corporate contractual obligations. Thank you, yes it is.
-If you've ever really worked with a kid, you know that getting down to eye level with them really helps to engage and motivate them. So stuff like this at their eye level is bound to engage and motivate them,
-What really concerned me is that there's a population of adults now who are apparently consuming this shit enough for there to be a rampant market for it...in the most liberal state in the United States. Either this anti-pattern was creeping up on us over years, or there's some serious money being pumped into this retrograde mindset that proposes the 2nd amendment meant for AR-15s to an inalienable right.
-I highly doubt it was a rogue decision by a single branch manager because, having flown to Minnesota and Wisconsin this week, I saw with my own eyes that this was a reoccurring trend, same thing in Detroit. Like a signal to people who would use these kinds of fire arms that they should stock up quickly. A reminder that you can only safe if you pack more than the next guy, that enough bullets will solve any problem. I can't even imagine what it's like in the southern states right now, maybe they're gearing up to stage a Confederate overthrow.
-It's the money that bothers me. To get this many separate magazine titles on a local rack, magazine not gun surprisingly, there have to be some serious contracts in place. This goes along with what the manager told a community member when confronted by the senselessness of this:
-
So vendors (of the magazines) must approve a change to floor placement in pharmacy chains? Sounds awfully like a well-known contractual obligation to me. Why would a gun magazine retail distribution contract stipulate that all gun magazines must be placed on the bottom rack? You must be 18 in order to become a legal consumer of a gun. Why wouldn't the magazines want to promote at eye level of their legal consumers?
-"Mommy, Daddy, why don't we own a gun?" is a powerful thing for a parent to hear. And because you can get gun loans with no background checks in some states, practically anyone can rapidly resolve that question in lethal force.
-I think the problem here is multi-level. Brands like Delta, United, MetLife, Bank of Omaha, Enterprise, Windham, and soon others publicly denounce the NRA while for-now-president Donald Trump and goons like Wayne LaPierre equally denounce gun control efforts. Figureheads aren't innocuous, they embolden an already primed generation. And on that...
-NRA and Video Games
-When I was a kid, video games were a cursed luxury. I paid for my own Famicom...I mean, Nintendo Entertainment System...and games too. Repairs even, when I spilled cereal into the thing (don't ask me to reconstruct that memory please).
-My conservative Christian parents begrudgingly lent me scant amounts of time on the family TV, that is until someone had the bright idea to put the NES on a small black and white TV that my recently passed Grampy had left to me. Spoiler: it was me. I am an engineer and things just occur to me sometimes. And I wanted to beat every goal that some big kid who made the game set for me. Inside there, I could do great things, certainly better than out in real life.
-In my 'Xellenial' generation, most parents either didn't understand video games or otherwise weren't the type to care about much else. I made sure that in my house, video games would be a family activity. The one's we chose are puzzle-solving, narration read requiring, fun incentivizing pieces of amazing art by teams of people who love what they do. While some people struggle to get their kids to care about books at all, ours read so quickly through everything we bring home from the library that they create their own stories, which are more than often very interesting.
-My partner and I have chosen to be very intentional parents. We feed our kids the best we can afford. We limit screen times to the weekends. We don't subscribe to religion. We are kind and sincere people. We will never give our kids phones because they will pay for them if it's important enough to them.
-So it's easy for me to view video games as a good thing in my house. That's not so kids who have been given technology and no constraints. The habit is formed to scratch the itch to get what you want, to win, to watch the next one, until it's done. Unconstrained screen time encourages obsessive-compulsive personalities in kids.
-Please just go to a GameStop and look at all the highest grossing titles. They're mostly first-person shooters like Call of Duty, Battlefield, Destiny 2, Wolfenstein, Sniper, Rising Storm, and get this..."Get Even", a story who's heroine has to shoot her way out of dungeons, office buildings, and SCHOOLS.
-Killing is not a game. Interactive killing stories are not video "games", they are early video "trainings". Take away the killing, like in Mario Odyssey, where multi-player and assist mode help various ages in a family all progress through problem-solving challenges together, and you have an actual video "game".
-[Note: listening to the Super Mario Odyssey background soundtrack, it's really great to hear that the musical director took the time to create an 8-bit version of the 44k version for the switch between 3D and 2D gameplay. That's like a little gift for me as a parent who grew up with these themes.]
-Titles that include violence should be barred from sale as "video games" and be regulated under federal law the same as movies and other adult forms of entertainment. To these video game vendors, you make blood money and you should be eradicated along with the mental disease you spread.
--"I would not squelch legislative efforts to deal with what is perceived by some to be a significant and developing social problem. If differently framed statutes are enacted by the States or by the Federal Government, we can consider the constitutionality of those laws when cases challenging them are presented to us” says U.S. Supreme Court Justice Samuel Alito. [ref]
A "Developing Social Problem"
-Is gun violence really a 'developing social problem'? Well, let's consider two facts: 1) if someone shoots someone else in public, that's a social problem and 2) gun violence is definitely developing.
-
There are other statistics that indicate that gun sales are going up while homicides by gun is going down. The margin variance in units per home is often plotted against a seemingly dramatic drop in incidents of violence. The type of violence matters if we consider what kids have been "trained" to do: pick up a weapon, find a way out of a building, kill what gets in your way. The mentally ill underage in this country have an easy onramp to becoming a statistic more like this:
-
Also look at: https://www.cnn.com/2016/06/13/health/mass-shootings-in-america-in-charts-and-graphs-trnd/index.html
--"The practices and beliefs of the founding generation establish that ‘the freedom of speech,’ as originally understood, does not include a right to speak to minors (or a right of minors to access speech) without going through the minors’ parents or guardians,” says Justices Clarence Thomas.
That's right, so when major pharmacies branded chains with managers that are members of my local community contractually decide to allow the NRA to engage with my children about owning a gun, when families allow children to play shooting games without limits, and when we don't confront these situations without hesitation when they arise, this is what creates a fatal national epidemic.
-What do we do about it?
-I drove to my pharmacies, grocery stores, and convenience stores to look for these displays. Find a few, tell them about the social movement and how they don't want that massive bad PR and inevitable sales dip next month due to boycott, and sometimes things change:
-
Still at eye level to a 2 year old, but at least the surface area for little eyes is reduced. I'd rather have Rachel Ray glaring at them than the muzzle of an AR-15.
-Local acts, local impacts.
diff --git a/_posts/2018-02-25-technical-recruitment-101-advocates-vs-evangelists.html b/_posts/2018-02-25-technical-recruitment-101-advocates-vs-evangelists.html deleted file mode 100644 index 4a62353..0000000 --- a/_posts/2018-02-25-technical-recruitment-101-advocates-vs-evangelists.html +++ /dev/null @@ -1,42 +0,0 @@ ---- -layout: post -title: Technical Recruitment 101 - Advocates vs. Evangelists -date: 2018-02-25 13:33:49.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- advocacy -- evangelism -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '1036' - _wp_old_slug: technical-recruitment-101-advocacy-over-evangelism - dsq_thread_id: '6504700513' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2018/02/technical-recruitment-101-advocates-vs-evangelists/" ---- -What's the difference between a technical evangelist and an advocate? What are these terms even? If you're a technical recruiter, I encourage you to know the answer to all of these questions by reading this piece for the next 4mins.
-What is a "Technical Evangelist"?
-A way to simplify the definition of an evangelist is to onboard people to their particular topic, do whatever it takes (hackathons, sponsorships, contribution, enablement sessions, flights...lots and lots of flights...blogging, interviews, code samples, research, t-shirt design, customer support, server setup). They wear all the hats, usually at the same time.
-What is an "Advocate"?
-An advocate defines their job based on the success of the customer (or anyone really) at the job they most need to do. Deep listening, directed dialog, metrics extraction, change impact quantification, being kind, and work framing are all tools in an advocate's toolbelt. It's caring about the other person first, then of course yourself secondly.
-Advocacy vs. Evangelism
-Advocates are good facilitators. Unlike evangelists, they don't assume that because a hat needs a wearer, that it must be them to wear it. They see the whole field, not just the ball in-front of them. Advocates identify what needs to be done cross-functionally and help to match people who want to and can do a thing to get that done. They think about the problem that people are trying to solve, and put resources into motion to reach that goal.
-Naturally, when I frame advocacy this way, it's easy to see why evangelism leads to burnout. Advocacy leads to faster burndown (e.g. sprint burndown charts) because facilitation, a scalability reinforcer, is at the core of advocacy. What problem are you trying to solve? What's required to get there? What challenges will arise? How can we most effectively help each other?
-How to Qualify Who Should Apply For Which Role?
-If someone demonstrates that they can differentiate between facilitating needs over taking on things as a personal responsibility to accomplish, you may just have the potential for an advocate. If a candidate lists off all the things they do without a clear definition of why first, you're better off funneling them into an evangelist role so they can either learn a better balance or burn out and find something else.
-Advocates build plans and often play a significant role in driving those plans to completion. Internal, external, independent, or consultative. When they do take ownership over a goal, you know that it will get closed, even if it wasn't achieved to expectation, it will get closed and retro'd.
-In Practice, I Say "Advocacy over Evangelism"
-Evangelists talk about their product and rarely take full responsibility for anything. They are often driven by others to do things, go places, speak under sponsored time, build samples, and be engaged with customers. It's right in the job title. "Evangelism" in Latin can be translated as "messenger". In other words, they have been told to deliver a message. Usually the messenger has nothing to do with the crafting process of the message, and would you want someone whose job is to talk rather than listen involved in what your message to people should say? I wouldn't.
-Put another way, an evangelist mindset attempts to frame the problems of the world into terms that their product can solve. Their focus is the product that they're incentivized to proliferate. If it doesn't quite fit a customer's situation, too bad, we'll make it fit. I don't know the job you're trying to accomplish because I didn't bother to ask, but I'm going to offer you a random solution anyway. And naturally, introducing a wrong-fit solution produces negative consequences (like lost time, lost money, and lost opportunity).
-An advocacy mindset exfoliates information required to make decisions about how to best accomplish a customer's true goal. Your product is not your software. Your product is a right-fit of people, process, and technology for a customer to successfully accomplish their goal. Your overlap to their situation in strictly wheelhouse terms might only by 10% or just one specific job. But if you can understand how important (or not) what you do is to completing the job they have at hand, then you can quickly fit what you offer to their needs.
-This is the heart of the way product teams should "go to market". They proceed as explorers and caretakers, not "disruptors" simply because it sounds cool. In this world, caring about how to help someone with their job better every day is disruptive. It's honestly disruptive, to sales, to marketing, to product management, and to vision holders in technology.
diff --git a/_posts/2018-03-01-fuel-for-your-brain-on-focus-usefulness-execution-learning.html b/_posts/2018-03-01-fuel-for-your-brain-on-focus-usefulness-execution-learning.html deleted file mode 100644 index b37428c..0000000 --- a/_posts/2018-03-01-fuel-for-your-brain-on-focus-usefulness-execution-learning.html +++ /dev/null @@ -1,53 +0,0 @@ ---- -layout: post -title: 'FUEL for Your Brain: On Focus, Usefulness, Execution, Learning' -date: 2018-03-01 12:38:28.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- advocacy -- IoT -- teams -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '1045' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2018/03/fuel-for-your-brain-on-focus-usefulness-execution-learning/" ---- -I hate acronyms. My dad used to use them far too much, the kind of guy that was more smart in retrospect than the kind of boy I was understood at the time. Kind, thoughtful, quiet, and invested in people around him.
-My current thought product is an acronym, "FUEL", based on a few key practices that I find are valuable to my current line of work as a Change Agent. These practices take time to develop and are only truly useful when used in parallel.
--
-
- Focus: ability to right-size activities to close the task(s) currently underway -
- Usefulness: ability to gauge effectiveness of work and reprioritize based on new ideas/objectives/activities -
- Execution: ability to match skill to task, collaborate the plan, and resolve blockers as they arise -
- Learning: ability to observe outcomes and refactor them into useful ways to improve all of the above -
Real-time Example of FUEL
-Last night after a meetup, I had a beer with someone I'd met before at a local conference but hadn't dived into. The opportunity presented itself, so I stayed a little later than I normally would. They are a CTO for a 50-person startup in town. Net-net:
-Paul: "What's weighing on you right now man, work related?" (L)
-Them: "Kind of glad someone asked...we have people issues." (E)
-Paul: "You're not alone...what kind?" (E/F)
-Them: "There are a few 'senior' engineers that don't produce like others." (U)
-Paul: "What's your plan for them and the rest of your team?" (U)
-Them: "We just laid off one after giving him a path, but the other two, I don't know...maybe add metrics, visibility...they're kind of SPOFs." (U)
-Paul: "...so you can quantify what you already know? How did we arrive here?" (U/L)
-Them: "They were here at the beginning, hence 'senior', but one guy hasn't committed code since Sept (5 months)!" (E)
-Paul: "Got it, they don't 'git' it. [laughs] How are you and the leadership team helping to coach other junior engineers?" (L)
-Them: "Well that's the problem maybe. We don't exactly have a culture yet, but our C-level relationships with each other are solid." (U/E)
-Paul: "I once heard that great leaders define their success by fostering other leaders. Do you know who your real 'senior' engineers are?" (F/L)
-Them: "Well I kind of already know who deserves the chance to step up." (E)
-Paul: "That's good, but not enough. People often hesitate on new things simply because they haven't experienced how it works yet. Your coaching needs to help those people get over any blockers to proving one way or the other if they can do the job well/right/better." (FUEL)
-Them: "I'm going to talk to our CEO about this. Can I get your card?"
-Paul: "Only if you intend to use it."
-
-Good enough exercise and learning for me for one night.
-diff --git a/_posts/2018-03-09-what-a-site-reliability-engineer-really-does-in-devops.html b/_posts/2018-03-09-what-a-site-reliability-engineer-really-does-in-devops.html deleted file mode 100644 index af98542..0000000 --- a/_posts/2018-03-09-what-a-site-reliability-engineer-really-does-in-devops.html +++ /dev/null @@ -1,60 +0,0 @@ ---- -layout: post -title: What a Site Reliability Engineer Really Does...in DevOps -date: 2018-03-09 22:13:27.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- advocacy -- DevOps -- equality -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '1063' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2018/03/what-a-site-reliability-engineer-really-does-in-devops/" ---- -
We really, really build ourselves into a corner with the internet and mobile and cloud and Agile "at scale". Good news is, we're engineers that can invent ourselves out of anything, or at least that's what's made all this money so far.
-What Is a Site Reliability Engineer?
-Srsly. Wikipedia. Too lazy? Fine, from Wikipedia (please donate):
--Site Reliability Engineering (SRE) is a discipline that incorporates aspects of software engineering and applies that to IT operations problems. The main goals are to create ultra-scalable and highly reliable software systems. According to Ben Treynor, founder of Google's Site Reliability Team, SRE is "what happens when a software engineer is tasked with what used to be called operations."[1]
What kind of this ninja trickery is this? Using common sense to make learn how to hire the best people in technology? Why would Google spill the beans on this hiring secret? Maybe they're sick of dealing with our broken shit.
-Our digital systems are ALL distributed and complex now. How can we still expect that having some ignorant code-jockey in a cubicle who never uses what they make control the entire business with the stroke of a keyboard? Because: we are cost-accounting brainwashed and forget that the job to do needs the right experience and skill to do it well. Meanwhile, we keep under-hiring operations and over-hire developers such that there's a 1-to-who-knows ratio between the people that press one button and the people that press another.
-If You're Offended By What I've Described, Congratulations!
-I am too. Things that are so complex no one person can understand them, those things are dangerous. Banking apps that aren't secure, mapping apps that get us lost and late, social media apps that show our kids their first porn, CGM devices that cost more in maintenance fees than their worth...it offends me when these things don't work. Technology that works is how I make sure I have money for a family, sponsored, biological, or otherwise.
-Our tech industry should be hiring people that can comprehend the things they deliver. People pay for things that work. If you don't care about others, at least you'll care about making money, and "right" software in a customer-obsessed market makes the most money.
-It's particularly offensive when the hybrid phoenix of a job title that 'Site Reliability Engineer' embodies goes largely unnoticed in high tech corporate mindsets. "What the hell is that, your latest professional title advancement scheme? Just because you mashed these words together doesn't mean you deserve a raise!!!" If you know the following things, you deserve a salary that rivals an enterprise VP of marketing:
--
-
- What your software should do -
- How your software does what it does -
- How to communicate the value of the things you're working on -
- Don't mind being woken up when it's broken for someone -
- Ignore those around you that don't think the above is relevant to do their jobs -
Go forth and make your first salary million in a few years, y'all who can. Do this well and grow.
-Why Do We See Site Reliability Engineering on the Rise?
-The tech industry is now at the point where we completely forgot that the persons who build software should know how to operate that software when it other people depend on it. Big money, consumer insatiability, customer centricity, and digital transformation has skyrocketed the imperative to make the modern enterprise business engine their engineering teams. We build shiny, complicated, and highly profitable things. What did we expect?
-We, the nerds, lured jocks in with our shiny things such as the Altair, BBS, and the entire mobile revolution...and they brought their friends. CFOs, 'professional CEOs', and other people that look at a hoodie like its pajamas that violate the corporate dress code. We allowed things to get this way #waterfall #agile #WomenInTech by being egotistical, lazy, impatient, and unkind. These are our chickens coming home to roost.
-And now we have to reinvent a way out of the 'shallow engineering' tech culture that looks skeptically at #DevOps as a management problem. I don't mean that everyone on your engineering team has to code, but the people who do code should understand the impact of what they do. This is ethical and this is practical. This is how you make your next billions.
-This is the new horizon for impactful, profitable, and scalable tech culture:
--
-
- Lisa Phillips, SRE @google -
- Taewha Lee, SRE @samsung -
- Jennifer Petoff, PM of SRE @google -
- Sally Lehman, SRE @Oracle for Govt. -
- Jennifer Kwentoh, SRE @aedcelectricity -
- Karen O'Connell, SRE @google -
- Mary Gardiner, SRE @google -
- Joan Smith, SRE at @twitter and @google (and Doctor Who fan, sup!) -
On #InternationalWomensDay, I guess that is all for now.
-diff --git a/_posts/2018-05-17-socratic-method-for-advocacy.html b/_posts/2018-05-17-socratic-method-for-advocacy.html deleted file mode 100644 index dc16f5b..0000000 --- a/_posts/2018-05-17-socratic-method-for-advocacy.html +++ /dev/null @@ -1,96 +0,0 @@ ---- -layout: post -title: Socratic Method for Advocacy -date: 2018-05-17 16:07:06.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- advocacy -- management theory -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '1073' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2018/05/socratic-method-for-advocacy/" ---- -
In disassembly of how I approach a zero-knowledge situation, a few key dynamics emerge. The goal of this simple framework is to accelerate the bond-forming process of dialog between individuals.
-Not everyone has an appetite for the full menu of techniques here, and right-fitting takes sizing up others in real-time, which itself takes practice. If you're really interested in how to have more effective conversations that leave all parties feeling positive, this is what I can currently offer.
--
-
- Introductions -
- Exploratory Listening
-
-
-
- Open-ended questions (a.k.a. "double-clicking") -
- Challenging to understand how topics relate and handling skills -
- Confirming what was shared with close-ended questions -
- Synthesizing the crucial point -
- - Identifying the "as-is" current state
-
-
-
- Share "the Summit" -
- Assess the journey to date -
- Materialize motivations (unearth drive) -
- - Planning the "to-be" desired state
-
-
-
- Identify milestones, past and future -
- Hunt for and prioritize gaps in order of risk to reach the summit -
- Summarize approach and obtain permission -
- - Co-develop a plan of action
-
-
-
- Goal-Strategy-Objective-Tactic -
- Visualize the timeline to reverse-engineer milestones -
- Socialize with key stakeholders -
-
Introductions
-Be cordial, clear about your role, short about your background, and quickly move to questions that help the other person(s) engage about themselves.
-Present well. Shave, brush your teeth, wear a bit of a smile. Smell like someone you'd want to be around. Be attentive, particularly in the eyes, and suppress the urge to look like you're anywhere else but this conversation right now.
-Make sure that the person has time to talk a little. If not, politely ask when. If so, ask them how they arrived here, where they're headed, and what's got them going this direction. Build a basic rapport in the first few moments. Start off well and build on the previous moment at every opportunity
-Exploratory Listening
-Ask far more questions than the number of statements you make. Extroverted learning prefers information you don't know over suppositions you could make alone.
-Use open-ended questions when you want to explore directions. If you don't know enough about what someone's describing, open more windows. When the way of conversation is unknown, let them talk and learn how they communicate. There's a lot of 'meta' information in human speech.
-It's important to challenge people, in no abrasive manner, but through asking how the current conversation point or branch relates to another topic or branch. The dichotomy of human behavior is that what is unknown represents simultaneously a source of both intrigue and fear. Questions can either encourage people to engage or to retreat, and our job is to engage.
-Asking a question that someone doesn't have an answer for leads to insight no matter what. How someone deals with unknowns will become useful later. When the question is right-timed and right-fit to the context, people without an answer are likely to explore with you. Poorly-timed or out of context, a question where no one has the answer feels awkward and often causes people to retreat.
-When something seems very important, narrow down the scope of the questions you ask, never coerce. Close-ended questions confirm that what you think you heard was actually what was said. Don't ignore non-verbal cues. Look for expressions of emotion surrounding quantities. Form a mental map of how these points affect their recipient and which seem relevant to them, not just you. Verify their relevance to others with close-ended questions.
-Once a branch of exploration is sufficiently developed, synthesize the crucial point uncovered back into the main theme of the conversation. By understanding it's impact, you can bring the point of the branch back to the goal of the conversation: to understand and support what the person is currently working on, suffering from, or driving towards.
-Anyone can voice the crucial point, but it's better if it's someone else. You don't have to be the one with an answer. This is why being curious about their perspective is so incredibly effective. Questions (open or closed) guide the conversation, even though people tend to think that an idea is original and their own simply because they voiced it. When someone thinks the idea is theirs, they tend to champion it with banners and bugles. Questions help steer champions in the right direction.
-Identifying the "as-is" current state
-The first move in any consultation should be to gain situational awareness. In other words, qualification of what dynamics and decisions are currently in place. Before a hypothesis and plan of action can be formed, observation.
-To make broad and holistic observations, you must share the summit. As the landscape of context emerges from your listening expedition and as you process that landscape together into a shared construct, a key state will emerge, "what we have accomplished so far." This is often coupled with pains and challenges regarding where the person currently is in their journey. Point and click with the camera in your mind because later we'll be doing a before-and-after photo collage. The highest point achieved is a summit, even if it's not the tallest peak visible.
-You may need to know more about how they arrived at their current summit in order to fully flesh out gaps in context. This sense is highly subjective and experientially developed, so practice earning the right to get to this point. It's important to step back and assess their journey every so often. This helps to uncover information gaps.
-These gaps of context, though you may be the one to bring them up, must be accepted by others before they can be addressed. You can't convince someone of a solution if they don't think there's a problem. And if something really isn't a problem, it's important for you to know that too. This is just another form of permission. The best way to ensure people show up to a party is to bring them with you.
-Equal parts collected context and 'meta' (appetite, capabilities, deficiencies) makes for a very good peep into what motivates someone, either to stay in the current state, migrate to a future state, or even drive a future state. Motivations are imperative to materialize about others. An accurate understanding of someone's motivations is worth more than developing a trust between you and them.
-Planning the "to-be" desired state
-Moving from what is to what could/should be the summit, start with our friend, an open-ended question, about where they want to be. In a group, this may get complicated; everyone communicates differently, and everyone has a different perspective. Quantize. Start with one person at a time. Give them non-verbal cues about length, but let them finish. The smaller the group, the easier this process is.
-With a shared vision of the future summit, ask what it would take to get there. Searching for important, measurable future achievements is called identifying milestones. Again, not something everyone knows up front, and while some people have immediate intuitions or ideas, others take time to form their thoughts and answers. Since humans are notoriously bad at predicting the future, extrapolating future milestones may be difficult. If stuck, revisit past milestones, the ones that brought them to the current summit. This will help form a 'meta' understanding about how people categorize just-busy-work from notable achievements.
-Make sure that the flow of questions to answers at this stage is 100% you questioning to 100% them answering. Questions may come up, but anything can be put in the "parking lot" provided it is suggested politely by you and agreed to by them.
-With a set of milestones laid out, reorder them based on which events are dependent upon others. Ask about which milestones enable other milestones. Build the most efficient decision tree. As always, pause at junctures that feel important to obtain agreement. For those that don't obtain consensus, circle back once, but otherwise parking lot it to keep the conversation flowing in the forward direction.
-Now that you have a map of the landscape and have sketched out a path to the new summit, hunt down gaps (unstated objections, parking lotted details, well-known deficiencies) to arrive at MECE (mutually exclusive, collectively exhaustive) on a list of variables that exist between knowns. There will always be another iteration, so "exhaustive" simply means when people seem like they are happy with moving on.
-With this list of gaps, ask how people would prioritize them. Root the conversation in terms of establishing priority based on relative risk to achieving the milestones and accomplishing the goal, getting to the summit.
-There is no one right answer so long as those in the conversation express their perspectives on what order things should be prioritized. The how and the why they prioritize things the way they do is not key to this logistics exercise. Though useful meta information about individuals, justifications and disputes drag down the goal, which is alignment and on an approach to confidently arriving at positive outcomes together.
-With a sense of how to prioritize milestones and gaps, circle back verbally to summarize the approach to moving from the old summit to the new one. This should take no more than 60 seconds. By now you should have mentally wordsmithed the goal, the milestones, and the approach such that you can do this. Those involved in the conversation up to this point will very likely unanimously agree because the summary incorporates their perspectives and terminology. When you do this effectively, you fade into their view of how to accomplish the goal. You are part of their team. You're in.
-Co-develop a Plan of Action
-With all this context and alignment under your belt, it's time to whip out some artisan management tools and have at it.
-Crystalize the new summit into a single statement/slide/executive summary structured as a Goal-Strategy-Objective-Tactic (GSOT) framing. This progression properly organizes information in an approachable hierarchy that stakeholders and sponsors can grok quickly. Everything in it will be defensible because it came from the people that know how to accomplish the new goal and they're already aligned with the details.
-There typically should be only one goal, otherwise, the approach serves more than one master and the purpose of the objectives become muddied. An approach (or strategy) should express a strong point-of-view about the objectives that is fundamentally different from the prior approach which led to the current summit, even if the prior approach was good at decision time.
-Every leg of the journey requires a pivot to ensure the direction is correct. Once a GSOT is constructed, make it easy for humans to visualize the timeline of objectives and key tactical events that support forward motion. These should be expressed in relative timeframes. Ranges, not absolute dates/times, are best at this point.
-Reverse-engineer where/when to place milestones progressively. Start by placing the biggest rocks in dependent-events order, then layer in tactics that support various objectives. Adjust as necessary when tactics are tight (risk of slippage) or clustered too closely together (work bottlenecks). Do this with one slide, and you have the most effective meeting with an executive sponsor you've ever had.
-Finally, with your ducks in a row, with everyone aligned, it's time to socialize with stakeholders one at a time. Don't present an idea for the first time in a boardroom. Objections are a fixture of the boardroom, so make it hard for them to maximize negative impact. Start with individual stakeholders. Do this under the auspices of collecting feedback, but then actually incorporate the feedback so you're not a liar.
-Pick your early stakeholder reviews carefully. A troll will turn around and proclaim your incompetence, but people with the most to gain will become your champion army when it comes time to fight the trolls. I wish you zero trolls, but living under bridges makes it hard to see them in advance.
-On a Personal Note
-The narrative above is a reflection of my observations and collaboration with technical engineering teams, boardroom executives, investors, sociologists, bartenders, and wolves. I broke my finger pretty badly two months ago which helped me deeply understand what would really hurt someone else. Likewise, as we exercise empathy and effective patterns of communication, we better understand how they improve us and their importance in our work together.
-Thank you for your time, and as you may guess, I'm also on a journey. We all are. If you found anything in this post useful, connect with me so we can journey a bit together.
diff --git a/_posts/2018-06-19-open-spaces-at-swiftfest-boston.html b/_posts/2018-06-19-open-spaces-at-swiftfest-boston.html deleted file mode 100644 index 4093f8c..0000000 --- a/_posts/2018-06-19-open-spaces-at-swiftfest-boston.html +++ /dev/null @@ -1,30 +0,0 @@ ---- -layout: post -title: Open Spaces at Swiftfest Boston -date: 2018-06-19 11:54:31.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: [] -tags: [] -meta: - _edit_last: '1' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2018/06/open-spaces-at-swiftfest-boston/" ---- -I'll be hosting an open spaces "in the wild" session at Banyan Bar from 6pm-9pm at the end of Swiftfest Boston Tuesday June 19th 2018. Look for me and for people with party beads like this:
-
Location and Timing -
We will be meeting at Banyan Bar right around the corner from the conference. Feel free to walk over after the conference ends at 6pm, the conversation will start at 7pm, and I'll be handing out free drink tickets at around 8pm after we are done. -
What is open spaces?
-Open Spaces is an "unconferencey" way to have small group conversations on topics that are most relevant to our community at just the right time. You submit topics (http://bit.ly/openswift2018) that you want to discuss in a group setting, then before the event, these topics are grouped into categories that help people self-organize into right-fit groups.
-What Will We Discuss?
-Because this open spaces event is the last workshop session of SwiftFest Boston 2018 (http://swiftfest.io/), it's likely that topics will gravitate towards technically-focused iOS development, testing, and operations, and team management. Great for conference attendees, great for Mobile Tea members too!
-However, a key affinity of the open spaces approach is to let the community figure out what matters most right now, and there’s a lot going on in the broader tech scene in Boston today, like DevOps leadership, sustainable startup practices, effective technical recruitment, municipal transportation and infrastructure support, etc. You never know, and that’s the fun part!
diff --git a/_posts/2018-07-02-how-to-hire-into-a-devops-market.html b/_posts/2018-07-02-how-to-hire-into-a-devops-market.html deleted file mode 100644 index 6c71d1b..0000000 --- a/_posts/2018-07-02-how-to-hire-into-a-devops-market.html +++ /dev/null @@ -1,67 +0,0 @@ ---- -layout: post -title: How to "Hire" into a "DevOps" Market -date: 2018-07-02 22:13:24.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- DevOps -- teams -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '1094' - dsq_thread_id: '6769378780' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2018/07/how-to-hire-into-a-devops-market/" ---- -Let's be clear. I'm mostly writing this so that I don't have to have another bullshit conversation with a high-level agency "technical" recruiter that doesn't really know what the hell DevOps really is. But before you misinterpret what's to come as a horn-rimmed trash session on recruiters (only a motif, I promise), consider that had you not read to the end, you would never have learned how to hire more efficiently, effectively, and ethically.
-Boston, We Have a Big Problem
-Aside from the occasional scuffle at a Java meetup, I'm in the Greater Boston Area, and Boston isn't exactly known as a shining example of the tech boom. Sure, we have Facebook, Amazon, ZipCar, TripAdvisor, Chewy, RaizLabs, and Pivotal amongst others. We also have Oracle, IBM, Salesforce, and a plethora of other institutionalized madness from the Fortune 500 which typically drags our communal technical proficiency rating down year after year. We also have some of the most dedicated, seasoned professionals which you'd be hard-pressed to find in Silicon Valley during a fictional OSCON meets AWS Re:invent meets RSA all rolled into one.
-Our problem is hiring. Everyone everywhere does not have the problem quite like we do. Its potency is not diminished in the DevOps space simply because its a generally applicable point to make for any highly skilled market. When something is as culturally intertwined as True DevOps mindset is in high performing teams, traditional recruiting approaches will always fall flat on their face. What people are calling DevOps positions right now are a loose collection of uninformed guesses, buzzwords, poorly crafted hiring pitches, philosophical paradoxes, voodoo, and outright corporate misunderstandings.
-Recruiter, Know Thyself
-What's required is simply a sea change in recruiting mindset: the fastest way to qualify people is for you to qualify yourself. I don't mean credentials, I mean the way you 'smell'. Not by your morning shower and shave, but by what real practitioners need to hear from your mouths in the first 30 seconds: "I've really been thinking about how to right-fit you and a few of my clients," or something equally real and challenging. How about "I'm interested to know what kind of work you'd like to be doing..." or "Which do you like better, people or code?" (that one is for all you recruiters looking to place the even more elusive 'DevOps Manager' position, which as it turns out is just a normal technical manager that's passionate about coaching and improvement and customers and building future leaders.)
-Or you could just leave us really good hires out of it, focus on the email overload flowing from opaque B2B recruiting firms currently choking your inbox (yes, I have recruiter friends and we talk), and then you'd be bidding for the lowest common denominators and margins, not the highest ones. Good luck with that approach, it leads to burnout and we DevOps people know a million ways to get burnt out. But unless you're ethically fine with placing unqualified people with unqualified teams for a buck, the first option is better for you and for everyone you churn through week after week.
-Be Transparent..to a Fault
-Recruiters and hiring managers, there is nothing worthwhile to hide from someone like me that isn't abundantly already transparent through what you don't know about the organization you represent. I don't have to ask questions about the org, just about you and your relationship with actual decision-makers, and what you don't understand or know enough about to hire on behalf of True DevOps teams properly.
-To get over cursory technical qualifications, ask people for examples of their work, or better yet look for them first; the simple act of a prospect answering you with examples of their expertise (or simply knowledge to-date) in particular areas is something unqualified people can't do and unmotivated people simply won't do. If your candidate hasn't used any of a dozen or more social platforms (like Stack Overflow, Medium, LinkedIn, etc.) to publish their own stuff, encourage them to so you can pass it along to the real decision makers.
-Once these minimum-viable qualification hoops are behind us, bring something to the table. Have a spine and a brain and a perspective about the challenge you're looking for my help to solve in an org. Understand the broader issues the organization you're representing is having, then ask us which ones we think we actually have the desire and a real shot at helping to move forward.
-Suspend disbelief for a moment...
-Drop the bullshit. Tell us who your client is when we ask, how many and which positions they have open, how many other candidates you're playing off each other for some high-dollar plot of glory, and for god's sake be prepared to describe your client's *engineering culture* like it was your own family. Go to a meetup every so often, or better, help to organize one. Sit through and listen, stop emailing from your phone through the presentations. Absorb what's happening, how people respond to certain topics, and if you don't find yourself startled awake by clapping, you may just learn something.
-Against all corporate recruiting conventional wisdom, when a potential candidate comes at you with such Maslowian questions as "Can you tell me a bit about their culture?" and "What challenges there would I even be interested in?", definitely don't talk about benefits packages (that are mostly all the same at a certain level anyway) and please don't use things like "occupational environment", "team synergy", or "interpersonal skills". Please be real, explain to us what you do so we can understand if you do it better than anyone else we will talk to that week, and leave the qualification script at the door.
-You're selling you to us first, then you earn the right to sell your client to us. If the order of those two things is in reverse, it's equally transparent where your placement priorities lie.
-Hiring Managers: Who's Doing Things Differently?
-Off the top of my head, I can think of a few folks in the local Boston DevOps meetup that are good examples of how to place highly qualified practitioners:
--
-
- Dave Fredricks, Founder, eninjia.io
-This guy is legit. Hard working, constantly advocating, and an early organizer of DevOps Days Boston, amongst other dedicated individuals.
-.
- - Sam Oliver, 3yrs self-employed recruiter, now FTE at PathAI
-This woman really rolls up her sleeves, as a co-organizer of the Boston DevOps meetups, and smartly carved out the "we're hiring" pitches from the "let's talk tech" conversation that this crowd is known to like as separate things. #listening
-.
- - Kara Lehman, Principal Recruitment Consultant, Huxley
-Her stock is climbing with me. We first chatted it up in 2017 and when I asked her what she was doing at the nerd-fest of a meetup at Pivotal, she said that she "needs to understand this thing called DevOps better", which is far more enlightened of a response that about 99% of other local recruiters.
-
If your name isn't in the above list, nothing personal, I'm writing this at 10pm on a Monday. Let's have a conversation where I vet you out and then maybe I'll write about you too. In the mean time, prepare yourself because it will be me who's interviewing you.
-Apologies and Thanks
-If you got this far, my gift to you is my true gratefulness for feedback you may have and maybe a retroactive apology for saying things harshly. We need to cut through bullshit, especially the corporate flavors of it, and this is my personal blog anyway.
-Recruiters: if you "can't do these kinds of things" in the position you're currently in, consider that you would probably earn far more by taking a new approach (like the one in this article) on your own and making 100% commission on your own closes instead of a measly 50k/year plus.
-Also, you can also reach out to me via LinkedIn and let's talk about your challenges in hiring, training, managing, or fostering a DevOps-minded team. I'm much nicer in person than in this post.
-More on DevOps:
--
-
- Folding Open Source into Enterprise DevOps -
- What a Site Reliability Engineer Really Does...in DevOps -
- DevOps Testing Strategy for Dummies -
- Socratic Method of Advocacy -
- How to Be a Good "DevOps Vendor" -
- DevOps is Organizational, Operational, and Orthogonal -
- Beyond DevOps: The 'Next' Management Theory -
Summary: personal aha...spend equal time on your community as you do on social media, and everyone will be better for it.
-The Local Buffoonery of Garden Variety
-When I woke, it was because of the shouting. That usually doesn't happen in my neighborhood at two in the morning. I don't mean like "hey, you forgot your beer on top of your car" kind of shouting, I mean the kind with curses and violence into the darkness of night. When I finally sat up to look down onto the street, between the fuzzy patches I saw a local taxi service driver furiously shaking his fist at something down the street yelling things.
-As he used his mobile phone to call the cops, there was time to lite a...non-uniform cigarette...and pace around right in the middle of the blindest corner in town. Four more real cigarettes later and five flashing, silent police vehicles quickly swarmed what's typically a peaceful daytime intersection that my apartment windows overlook. Two at a time, the tatted-up Ford Explorers sped away in the fisterly direction, leaving the final officer to take what I can only imagine was a confusing deposition and also a flashlight look-around to validate the driver's story.
-Today, Sunday, I drove downtown to ask the daytime taxi dispatcher what happened. I was expecting that I'd also have to stop up at the police station to get their official statement of what happened, but the guy at the desk knew the whole story and was happy enough to spill. Here it is.
-After an overly complicated girlfriend-plus-other-friend pickup, some mentally imbalanced passenger, seeing a fare that surprised him as far too high, escalated an argument about the fare to an unexpected roundhouse punch to the driver's head. That's right, with two other people in the back, he temporarily disabled someone with the responsibility of driving them all somewhere on Eastern Point.
-How doped up or insane does a passenger have to be to attack the driver? Why is this ragtag group of weirdos proceeding to somewhere on Eastern Point (very affluent)? Movie stupid, very upset, either or both. The world is really, truly crawling with crazy, fucked up people at all levels (especially in politics). And, like taxi drivers, sometimes you don't have the economic luxury of pushing back on psychotic behavior when it comes your way because of what you do.
-The Political Buffoonery of Friends and Neighbors
-Assault is of course a crime...depending on who you are these days. It should be, it doesn't work at very large scale in the commons where most of us live. But for some reason, everyone from the well-educated to the god-fearing seem to think that it's open season on assaulting others who don't share their beliefs. When it happens verbally on local social media (in particular, Facebook), I step back and ask, "what side of all of this do you want to be on?"
-Take this guy I'd hope to call a friend, Jam. Jam is all kinds of good and crazy and well-intentioned. He runs a local blog, or maybe a cemetery where overly opinionated blog posts go to die, I don't know at this point. Whatever he is, he's complicated. Sadly, and rightly so, the current political climate has turned him into an-eye-for-an-eye asshole on social media. A counterpoint-liberalist, we'll call him the Cook, recently said some things in response to Jam and other local liberals' boycotting a local business for some insensitive things the conservative owner said. This is like Spanky and Our Gang vs. the Little Rascals type bullshit. "Not beef", as some might call what this is.
-This unwelcome display of reverse-perspective outreach sparked a knock-down, drag-out social media war between two individuals that are very reasonable when not provoked. But neither even saw what they were doing: focusing on the meta over the impact. Impact of their meta, more polarization. Opportunity cost of the meta over of useful action. Long-term impact of rejecting others' views and position on their journey, a.k.a. karma. Acceptance that there is a shared journey. Acknowledgement that they may be wrong, no matter how much they think they're right. That they are most certainly wrong if they think they're the only one's who can see something right. Complete subjectivity instead of focus on objective measures.
-The result is that at least one of these parties is regretful and walking around hurt by it all. The worst thing is that I don't think it's my friend Jam. He still hasn't worked his ego problems out before diving headlong into 'other' advocacy with women's movement and anything else the overwhelmingly liberal-biased feeds feed him. All those things are good, but without love and respect, what good does it do for Jam to constantly interject himself?
-It's all buffoonery until we do something useful with others, not flashy or geeky or memorable or unusual. Not something that gets you a group photo with Captain Picard at a fancy inherited wealth house or the following to argue to against Trump multiple times a day instead of showing up to community roundtables about how to improve local education. Speaking from this experience, it was nice to be in a place where everyone practiced listening as much as they practiced having a spine and a heart. This is what I and others did yesterday from 3-5pm downtown. I don't feel at all that way about the online shouting match Jam and the Cook participated in.
-Don't Let Nationalism Buffoonery Legitimize Local Buffoonery
-You may think, "I'm so different than what's going on politically right now, I need to speak out!" Please find more useful ways to argue with people over social media. Need solid reasons not to? How about that you won't be permanently quoted in exquisite online conversational detail by an article like the one you're reading now. How about, instead of counter-trolling, you could be sipping something nice right now. Fuck, anything but arguing with neighbors online.
-Instead of escaping from the real world, think about how the way you view things affects those around you. How about that you don't want your check-in with captured moments of friends and family to be perforated by "what's trending" bullshit between town neighbors. Sure, share stories and articles that you've validated and make sense to you in various forums; but once an online conversation looks like it's getting nasty, politely move along. Don't let something as transparent and egotistical as "my blog swings 2,300 local votes" be your hood ornament. If people are being victimized, eject the violator and report their poor behavior.
-In short, and to bring it full circle, don't let a culture precipitated by a talentless, washed-up genital wart of a President and his cargo cult nationalist, supremacy-retrograding base suck you in to the human-capital machines that Jack Doorsey and Mark Zuckerberg built but now take it with their pants on from Russian mafia-state. Find me instead and we'll identify something far more useful in our town to work on together. I've been collecting people's challenges, many of them just take a little bit of help from others to overcome.
-If you can't be bothered to focus on useful activity over timespend on social media, at least try this: every minute you spend on social media this week, spend equal on something useful for the people who live around you. Can't think of what that might be? I've got some ideas handy:
--
-
- Donate supplies to a program that need it (this week, it was tampons) -
- Walk around your neighborhood (or just your block) and pick up trash -
- Clean up, repair something in, or just post pictures of a great little local park -
- Engage others to write about their local views over just your small clique -
- Prioritize and facilitate actionable community projects needing nothing more than hands and time to complete -
When I see people not doing stuff like this, it's usually the folks that maybe don't deserve a seat at our community table. They like to bark, but don't like to dig. They look influential, but are really inconsequential. They are simply taxpayers, not community members.
diff --git a/_posts/2018-09-16-engineering-is-about-more-than-code.html b/_posts/2018-09-16-engineering-is-about-more-than-code.html deleted file mode 100644 index 169e3dd..0000000 --- a/_posts/2018-09-16-engineering-is-about-more-than-code.html +++ /dev/null @@ -1,34 +0,0 @@ ---- -layout: post -title: Engineering Is About More Than Code -date: 2018-09-16 09:54:37.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- advocacy -- DevOps -- DevOps Days Boston 2018 -- software -- teams -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '1131' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2018/09/engineering-is-about-more-than-code/" ---- -Curiosity is what drives engineers, and is equal parts curse and companion. An engineer isn't limited to development or operations. An engineer would be a problem-solver in both areas, probably more. Curiosity is a surprisingly rare quality in people, even in technology.
-If you want to know how something works, take it apart and observe. My first digital systems disassembly was a Sony Discman in 1989. Whatever I did, I fixed it. The feeling was powerful. It just took me 25 years to realize that there are many broken things in the world and to prioritize which one's I involve myself with. Understanding the problem is crucial.
-This is how I approach many conversations, navigating purposely and politely until there's a useful reframing. People aren't things, so be kind, be sensitive, and be patient. When you engage, learn about their biggest challenges, how they approach things, and what drives them. Just start with that.
-[If my 80's discman was still around, it would be like "yup, those were the days". My 8086 XT clone next to it would splutter out some op codes. My Mega Man game watch would be waterlogged and stuck in a loop. Which all lead me to the next action...]
-Put things back together again so that they work, hopefully, better than before. It's just courtesy. In commit-worthy code that's called hygiene. In conversation, that's called maintaining a shared view or vision. Do these things enough and you'll find that the way to a common goal is easy easier than with clutter obscurring your journey, and for others'. When you can row in the same direction, you get to your destination a whole lot faster.
-Put it with other things to see where it doesn't work. That's integration and it's not always easy, especially if it's your own code or new auto-scaling configuration that causes unforeseen things to blow up. Get to know what your thing does before and after you put it out in the wild. Be honest with yourself and others about the time this takes.
-There are some things you can take apart, and some things you can't. If you can't or if it's too much effort for not enough value, move on to another learning tool, but remain committed to your goal. In the light of fundamental flaws in how we think about security, privacy, and basic human welfare right now, seek something you're proud and grateful to do. [There are some very worthy things happening in Boston right now.]
diff --git a/_posts/2018-10-14-adhocathon-selenium-conf-chicago-2018-rstats-in-grid.html b/_posts/2018-10-14-adhocathon-selenium-conf-chicago-2018-rstats-in-grid.html deleted file mode 100644 index a634be9..0000000 --- a/_posts/2018-10-14-adhocathon-selenium-conf-chicago-2018-rstats-in-grid.html +++ /dev/null @@ -1,54 +0,0 @@ ---- -layout: post -title: 'Adhocathon: Selenium Conf Chicago 2018 - Rstats in Grid' -date: 2018-10-14 12:31:29.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: [] -tags: [] -meta: - _edit_last: '1' - _oembed_d148797de1b1c95b481aa29648bec0d4: "{{unknown}}" - dsq_thread_id: '6969692247' - _wp_old_slug: adhockathon-selenium-conf-chicago-2018-rstats-in-grid - _thumbnail_id: '1144' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2018/10/adhocathon-selenium-conf-chicago-2018-rstats-in-grid/" ---- -The goal of this time-boxed, scoped hackathon is to visualize the health and activities of a Selenium Grid instance by contributing the code necessary to report key metrics via Statsd. This supports observability and operational reliability requirements of continuous testing "at scale".
-Date/time: Thu Oct 18th, 6-8pm in the LaSalle room
-Sign up here: https://goo.gl/forms/SztY8j7j83RZQEkJ2
-Why Selenium Grid and Statsd?
-Out of the box, visibility into how Selenium Grid is working/performing can be...limiting. This is a problem, because when a test fails (maybe multiple times), how can you quickly know where the issue originated (app, test, environment, data) if all of these things are not easily observable?
-...and because lots of people use both of these technologies, or at least should, and they are both premised on separation of concerns at architecturally significant points, such as test-vs-environment and business-logic-vs-logging. Also, this is a stated potential area of focus for improvement by a few of the Selenium core contributors, and because the Statsd protocol has extremely wide-spread support in the industry.
-Why So Last Minute at the Conference?
-A few weeks ago while helping to organize DevOps Days Boston, I had a mental breakthrough about why testing gets talked about very little in organic DevOps communities: much of automated functional and non-functional testing doesn't speak the same lingo as development and operations.
-One of the core DoD organizers, Laura Stone, had just published a set of training videos on Statsd which I was impressed with, learned something from, and it struck home that we don't have telemetry for Selenium Grid, a focus of my recent pet project during the day.
-Many large organizations I work with use WebDriver extensively, and in order to fit something as critical as functional testing into a DevOps-minded delivery pipeline, software components and services must be operationally visible. The health of supportive dependencies, as they contribute affinities and deficiencies, are often as important as the health of primary deliverables.
-So I had the courage to reach out to my volunteers' contact at Selenium Conf Chicago about an idea I had on contributing to the Selenium project, namely layering in Statsd functionality into Selenium Grid to improve the observability.
-Why Is This Not a Hackathon?
-My ideas are shit. Not always, but most of life is an opportunity to learn what you don't know. A majority of hackathons typically structure as a competition for prizes, and typically gravitate around some narrow vendor offering or are staged solely to promote some new service. Some business sets the tone and parameters, then judges contributions based on a set of meritable characteristics. This is not what I want.
-In every area of my professional life, I look for opportunities to listen, learn, and deliver value. What better way to do those things that get a lot of other people to do them with you, multiplying the chances of exfoliating great ideas rapidly! The closest I know to this idea is a hackathon, but much of the work to getting the best ideas out quickly isn't coding; it's necessary, but often not the most relevant outcomes. So I needed a new name: "ADHOCATHON" is an iterative, time-boxed design-code-deliver session format.
-How Is It Going to Go?
-I have no idea. I often get these things right, like last month when I got 40+ managers together as a meetup to have a "forward dialog" about hiring and recruiting challenges and tips in the current Boston tech climate. The point is not that you know what to expect, but that you can facilitate the "best" outcomes to become a useful contribution.
-That said, the conference organizers were happy to help however they could, and now we have a room at the venue right next to the first-day reception area. I have a number of customers and colleagues signed up to be there. There will be tweets going out via the conference handle. And I will have found my way into the place that Tim O'Reilly describes as "creating more value than you capture".
-For the next few days, I will be scrambling to provide minimalistic resources to accomplish this goal, such as:
--
-
- A temporary code repo of example code I've written and fought with to date:
-github.com/paulsbruce/selenium-server-statsd
- - A slide deck for the multi-cast screen share at the beginning of the session:
-https://docs.google.com/presentation/d/1Xo_79lcQ-_FXy0GtJM0D2akFrk-EGK3UWGj2gFq6wsE/edit?usp=sharing
- - Snacks and maybe drinks -
- Venue logistics, set up and teardown, and other checklists -
- Post-hocathon stewardship actions to formally contribute our efforts -
Whatever happens, it will definitely be interesting. I'll publish a follow-up article on what happened and outcomes here after the conference ends.
-diff --git a/_posts/2018-10-16-alldaydevops-2018-progressive-testing.html b/_posts/2018-10-16-alldaydevops-2018-progressive-testing.html deleted file mode 100644 index 0ca9c3d..0000000 --- a/_posts/2018-10-16-alldaydevops-2018-progressive-testing.html +++ /dev/null @@ -1,40 +0,0 @@ ---- -layout: post -title: 'AllDayDevOps 2018: Progressive Testing to Meet the Performance Imperative' -date: 2018-10-16 23:17:19.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- DevOps -- performance -- testing -tags: [] -meta: - _edit_last: '1' - _thumbnail_id: '1151' - _oembed_6210b6efcc83f7f69ca35f17e55ae346: - _oembed_db85fc3f681f5d30a58f201b2c15e73f: - _oembed_time_6210b6efcc83f7f69ca35f17e55ae346: '1539773708' - _oembed_time_db85fc3f681f5d30a58f201b2c15e73f: '1540228928' - _oembed_0d4f7d3247600b34c550a04218b98157: "{{unknown}}" -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2018/10/alldaydevops-2018-progressive-testing/" ---- -
Mostly as an appendix for references and readings, but whenever I can I like to have a self-hosted post to link everything back to about a particular presentation.
-My slides for the presy: https://docs.google.com/presentation/d/1OpniWRDgdbXSTqSs8g4ofXwRpE78RPjLmTkJX03o0Gg/edit?usp=sharing
-Video stream: http://play.vidyard.com/hjBQebJBQCnnWjWKnSqr6C
-[embed]https://www.youtube.com/watch?v=DexfpnbBFn8&t=15995[/embed]
-A few thoughts from my journal today (spelling and grammar checks off):
-
Will update as the conversation unfolds.
diff --git a/_posts/2018-12-20-wanting-vs-having-for-my-7yr-old.html b/_posts/2018-12-20-wanting-vs-having-for-my-7yr-old.html deleted file mode 100644 index d57371a..0000000 --- a/_posts/2018-12-20-wanting-vs-having-for-my-7yr-old.html +++ /dev/null @@ -1,49 +0,0 @@ ---- -layout: post -title: Wanting vs. Having for My 7yr Old -date: 2018-12-20 17:39:40.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- parenting -tags: [] -meta: - _thumbnail_id: '1175' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2018/12/wanting-vs-having-for-my-7yr-old/" ---- - -I make a lot of mistakes. I try not to do it on Github where commits are a permanent record of your competencies. So are children, only the impact of a word said harshly or missed opportunity for positive reinforcement take immediate effect and often have lasting effects on their lives.
- - -Taking my son and daughter out one-on-one to get gifts for the people in their lives is important to me, and despite sometimes feeling like it in the moment, is never a mistake in the ideal sense. Having a positive experience doing so is also part of a successful outing, because if it's a negative experience, the whole thing is useless.
- - -A few weekends ago, I went out with my daughter. When we finally ended up at the toy store to get something for her brother, naturally she found something for herself. Because she's seven, and no matter how zoned in of a quick pow-pow we had outside before entering about who she's here to shop for, she still looks at the world with her-eyes. Arguably we all do this, regardless of our age, defaulting to looking out for number one in all manner of situations. That day, it was a cheap, pink, plastic set of glam rings.
- - -I said no, but I'll think about
In life, the moment you want a thing and the moment you get it are often different.
- - -If I had bought her that thing, what next? Should she have it right then, as inevitably she would. This is just kicking the 'delayed gratification' can down the road. And in juxtaposition with the notion of immediate positive reinforcements, her want wasn't connected to a prior goal or reward plan. So no, I don't just buy things because my kids want them, even when I can. It's not just the principle that matters, my choices (a.k.a. commits to their codebase) have an immediate impact. The manner in which I roll those decisions out matters as much as the very moment of desire in their hearts.
- - -So what will I do with this new hypothesis? As with all hypotheses, I will test it. On the morning of the Big Day, after all the things are unwrapped and people chill the fark back down and people have 2nd coffees and there's bandwidth for sharing that learning, I will say to her:
- - -"Love, I too seek the desires of your heart and will always support you in them. In life, the moment you want a thing and the moment you get it are often different times. Remember this thing, what you wanted weeks ago? Well I remembered and it's been waiting for you the whole time. But there's a time and a place, which is today! Trust in me, one who has gone before, that I can help you learn to deal with the waiting time between when you desire something and when you achieve it. Remember that my love for you is something you don't always know how to see, but never doubt that it will always be there."
- - -
Foreward: Since I highly doubt the following concepts will see the light of day in the final draft of IEEE 2675, I wanted to document that in
What is a 'value chain'?
- - -A value chain is a set of activities that a firm operating in a specific industry performs in order to deliver a valuable product or service for the market. The concept comes through business management and was first described by Michael Porter in his 1985 publication, Competitive Advantage: Creating and Sustaining Superior Performance.[1]
- - -The idea of the value chain is based on the process view of organizations, the idea of seeing
[Wikipedia: Value Chain (Porter)]
- - -How does it related to IEEE 2675?
- - -As related to DevOps, the Value Chain for Software Delivery is an application of a lifecycle perspective that scopes standards adherence to only certain individuals based on their participation in primary or supporting activities related to a particular software product.
- - -DevOps is about continuously delivering value to users/customers/consumers. A value chain perspective disaggregates activities from an organizational funnel, making it easier for teams and consumers of a particular product or service to ask "does this thing meet the standard" without unintentionally including other unrelated teams or products, were the question be phrased as "does this organization meet the standard".
- - -What problem are we solving by using 'value chain'?
- - -Adoption. In large organizations with many independent groups or product teams, DevOps principals may apply across the value chain of one product, but not another, provided these products are completely independent
Examples of where the broadness of using “organization” presents a challenge to adoption:
- - --
-
- Consulting agencies with many independent project teams on working on separate contracts/products
-
-
-
- If only one of those contacts require adherence to 2675, does this require every team/contract (both in the future and retroactively) to do the same? -
- Using “value chain” would scope 2675 adherence to any and all parties performing activities germane to delivering that specific contract/product -
- We know that if even one team implements DevOps per 2675 and sees success, organizations are likely to grow that out to other teams over time; “value chain” helps adoption of the standard. - -
- - Enterprises in the midst of transformation to DevOps
-
-
-
- Can they claim adherence on a specific product if the “whole organization” can’t yet? -
- Much like the above agencies argument, when the scope of adherence is based on activities relating to delivery of a project, enterprises are far more capable of becoming “DevOps ready” because they can grow the practice out *over time* -
- In DevOps, key success factors are determined – driven – by customers, the end user. - -
- - Organization’s requirements from non-technical internal agencies
-
-
-
- Can we expect that the legal or HR departments are also “DevOps”?
This would of course need to be defined by activities that support the delivery side of the business (i.e. billing, purchasing, etc.), begetting an activities-based perspective on implementation over organizational labeling
-
- - Can we expect that the legal or HR departments are also “DevOps”?
Why is this of importance to IEEE 2675?
- - -In layman's terms, adoption of IEEE 2675 at an organizational level can't happen overnight, especially in large enterprises with many teams. Fortunately, it doesn't have to, provided we adequately scope 'shall' statements with a perspective that A) is reasonable in scope and impact on the org, and B) enables parties to agree on what it means for a software product to have been developed and delivered using this standard.
- - -How and where would we use 'value chain'?
- - -Places in the text where use of the word 'organization' could infer that IEEE 2675 must be implemented across the whole organization before any single team or product could claim adherence to the standard. For instance:
- - --
-
- Too broad:
-
"Organizations shall implement effective continuous delivery procedures aligned with the architecture definition in a manner that meets the business needs of the system." ... "DevOps itself requires effective architecture across the organization to ensure that application build, package and deployment procedures are implemented in a robust and consistent manner." (6.4.4)
-
- - Scoped:
-
"Organizations shall implement effective continuous delivery procedures within a particular value chain that are aligned with the architecture definition and in a manner that meets the business needs of the system." ... "DevOps itself requires effective architecture across the whole value chain to ensure that application build, package and deployment procedures are implemented in a robust and consistent manner."
-
- - Too broad:
-
"Organizations shall maintain an accurate record of both code deployed and fully automated mechanisms in order to remove obsolete code."
-
- - Scoped:
-
"Organizations shall maintain an accurate record of both code deployed and fully automated mechanisms in a value chain in order to remove obsolete code."
-
Additional references:
- - --
-
- Value Chain for Software Product Delivery, Thomas Lee, 2015 -
- Wikipedia: 'Value Chain' -
- The Software Value Chain as an Analytical Framework for the Software Industry, Pussep/Schief/Widjaja/Bucksmann/Wolf, 2011 -
- A Value Chain Analysis of Software Delivery Components, Boehm/Papaccio, 1984 -
- Mapping of Porter’s value chain activities into business functional units, Saha, 2011 -
- Competitive Advantage: Creating and Sustaining Superior Performance -
A few words to manufacturers and vendors of tech toys: to really be ready for the holiday, if your product requires software updates in order to work or is in any way internet connected, make sure your site stays up. Otherwise, you just shipped coal.
- - --
-
- Provide more than one distribution point for downloadable updates/binaries -
- Rely on CDNs for static assets (like installers and documentation) -
- If your update process must rely on live services, make sure they're scalable
-
-
-
- Load test subcomponents/microservices AND the end-to-end process -
- - Be prepared for damage control by:
-
-
-
- Monitoring site uptime and availability to know when things are broken -
- Proactively establish a communication channel with customers during issues -
- Properly staff IT and support for during AND post-season issues -
- - Make sure the cost of downtime is factored into your next sales cycle -
Santa Brought Us a Brick?
- - -Let me start by admitting how 1st-world this example is. Robots as play-things are still not exactly 'so easy, a child could do it', and Roomba's have been around for almost two decades, but we still have yet to see a really down-to-earth home robotics project that really works for children under 10. I don't just mean toys that are not pre-assembled, even the right-out-of-the-box kind often require firmware updates or online services to really work as expected.
- - -Case in point, the Meccano MAX. Though it only took about 3hrs total to put together, this morning when we finally turned it on and went to connect for a firmware update, the vendor's website was down...hard. The instructions said, before anything else, update the 'MeccaMind' and voice commands weren't working without, so, blocker.
- - -As an ops nerd, I slapped an uptime monitor on it to know when (if ever) it was back up:
- - -
That didn't stop the whole multi-day experience from deflating to a dud. We all worked on this thing together and then before it can do anything, we are stuck guessing about when we can actually enjoy it. Don't blame Santa, blame the geniuses at Meccano.
- - -Performance, of your product, of your service, of your site, is imperative to delivering what you sold people. Availability, uptime, scalability, and reliability matter by default now. Everyone has downtime, but 4hr recovery time on your corporate domain isn't just irresponsible and costly, it's plain embarrassing...and transparent.
- - -Why Is This Even Important?
- - -My 7yr old is crazy into coding right now. Granted, we use a visual Code Block Editor mostly for lights and tones, but it's a great way to introduce concepts like flow control and formal logic. As soon as she saw an example I built that used function blocks to encapsulate and reuse logic, she instantly understood and started refactoring her programs.
- - -But finding the right project for varying stages of aptitude, appetite, and enjoyment is a real challenge. No thanks to marketing, but also unanticipated road-blocks like service and subscription dependencies are hard for consumers to factor in when purchasing. Even when you do find a right-fit project, if bone-headed problems like website downtime occur, it can become a negative experience for the child (or student).
- - -It's important for STEM product manufacturers and software vendors to really think about the impact of what they're selling, how they're delivering it, and how to support people who paid them money for something to accomplish a goal. If you don't have the optimal consumer's experience in mind, it will eventually cost you.
- - -Need the Robot Software Updater?
- - -I can't archive everything on their site, and it's their job to provide reliable content distribution, but in case you find yourself stuck like I was, here are links to at least the firmware updater tool:
- - --
-
- Windows: RobotUpdaterSoftwareV1.28.5.zip -
- Mac: RobotUpdaterSoftware_V1.23.pkg -
Also, never ask a consumer if they want to choose (null).
- - -
And if the updater gives you flashbacks to DirectX drivers from 1997, don't worry. It only looked like it bricked my robot for about 4mins before providing UI feedback:
- - -
-
Once you do get back to the modern era, a 98.6MB mobile app to control it shouldn't be too hard on your data plan. They also need to know your GPS location, phone contacts, and file storage for some reason.
- diff --git a/_posts/2019-03-25-ddos-prevention-and-mfa-hardening-before-release.html b/_posts/2019-03-25-ddos-prevention-and-mfa-hardening-before-release.html deleted file mode 100644 index 7fe2249..0000000 --- a/_posts/2019-03-25-ddos-prevention-and-mfa-hardening-before-release.html +++ /dev/null @@ -1,39 +0,0 @@ ---- -layout: post -title: DDoS Prevention and MFA Hardening Before Release -date: 2019-03-25 07:32:35.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- automation -- DevOps -- security -- testing -tags: [] -meta: - _thumbnail_id: '1207' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2019/03/ddos-prevention-and-mfa-hardening-before-release/" ---- - -Thanks for coming to my session at DevOps Days Pittsburgh!
- - -This page will be a placeholder for slides/video after the presentation.
- - -Resources:
- - - - diff --git a/_posts/2019-05-17-three-sixty-five.html b/_posts/2019-05-17-three-sixty-five.html deleted file mode 100644 index 6298069..0000000 --- a/_posts/2019-05-17-three-sixty-five.html +++ /dev/null @@ -1,69 +0,0 @@ ---- -layout: post -title: Three, Sixty, Five -date: 2019-05-17 21:11:16.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- advocacy -- teams -tags: [] -meta: - _wp_old_date: '2019-05-19' - _thumbnail_id: '1218' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2019/05/three-sixty-five/" ---- - -In less than three days, I recently passed through five airports using six planes to meet with three very important teams. Not a single Uber (because fuck Uber), 14 meetings/calls later, and a collective 9 hours of sleep. Before I let the normal hurricane of work-life sweep these events away, I want to distill learnings.
- - -First...Context:
- - -In 2017 after an exhaustive job search, I found myself talking with an SMB CEO about what my goal was. I said then, as I still say now, that one of my primary goals is to “be in the room”. Rooms where decisions are made; not only internal decisions, external ones too. You have to prove your value at the table in these rooms, and this is what I do.
- - -Three
- - -During these past three days, three very meaningful things happened.
- - -One: an F100 customer I've invested a ton of effort in for the past 6 months demonstrated how impactful that work has been to their strategic quarterly planning in front of two very important people in the full-time company I work with. This took me by surprise, but is exactly what happens when you're doing things right (I think). People benefit so much they want to share it with others. I see this as a positive characteristic of humanity.
- - -Two: another F100 prospect I've invested a ton of time with confidentially shared how the internal dynamics of their big-picture financial strategy over technology funding underscores the importance of having a flexible approach to the acquisition process for software vendors and GSI partnerships. I would never have had the opportunity to learn this in school, at prior software jobs, or without the support of an amazing business team (incl. key individuals in leadership, partner channel, sales, and product management).
- - -Three: I got another very private F100 customer to discuss patterns, practices, and industry trends with a senior researcher at a major analyst firm. Early in my career and for many people in what I refer to as 'shallow engagement work patterns', this would be impossible; but I believe in value-first relationships, referring to how Tim O'Reilly puts it: "create more value than you consume". Really doing this with your customers takes more than an ask, it takes gives which make asks pale in comparison.
- - -Sixty
- - -These and other things happened the past almost-three days, but took sustained efforts. The same statement applies every day inside my day job. Some things happen overnight. Some things aren't under your control. That leaves the universe of possibility to things that happen over time and are totally under your control. This is very good news for us "busy brains".
- - -Self-control is a characteristic people pick up on subconsciously, but lack thereof most certainly triggers conscious flags. I have a few habits…nail-bitten fingertips, post-its written everywhere, late-night emails…that I think qualify for the latter. In contrast to the last sixty hours, I'm going to scale back these and other habits over the next sixty days. This will take effort, sustained and supported with the help of others.
- - -A colleague said to me last month: "don't wait for your doctor to tell you". Habits are something that can be changed. Trust takes time to build and can be lost overnight. Observing people's habits can accelerate identifying trustworthiness. Being someone I know is trustworthy, I want to seek ways to accelerate others' trust in me. Controlling habits is my immediate actionable.
- - -Five
- - -If you’re ambitious, curious, and capable, tech is a fantastic sector to be in. Things move very fast, money too. Five years ago, I very deliberately pivoted my career from writing code all day to observing and directing the code that people operate by. In retro, I could have done this in my early twenties, but at least it's happening now.
- - -Since 2018, the rooms I’ve gotten really good at working in are typically at an F100. Under the surface, things in F100s are so complicated and fucking messy…I love it when things are messy…but on the shiny topside in rooms with egos and initiatives and collateral on the line, dialogs have to be squeaky clean. The trick is how to bring the room together with anything but the mundane, with active listening, informed point of view, inspiring next-challenges, and most importantly, sincerity. Effective communication and collaboration takes experience, and experience takes doing. This is what I'm doing all the time now, honing and refining.
- diff --git a/_posts/2019-06-01-performance-engineer-vs-tester.html b/_posts/2019-06-01-performance-engineer-vs-tester.html deleted file mode 100644 index 5dc28b1..0000000 --- a/_posts/2019-06-01-performance-engineer-vs-tester.html +++ /dev/null @@ -1,87 +0,0 @@ ---- -layout: post -title: Performance Engineer vs. Tester -date: 2019-06-01 23:43:37.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- DevOps -- performance -- teams -- testing -tags: [] -meta: - _thumbnail_id: '1229' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2019/06/performance-engineer-vs-tester/" ---- - -A performance engineer's job is to get things to work really, really well.
- - -Some might say that the difference between being a performance tester and a performance engineer boils down to scope. The scope of a tester is testing, to construct, execute and verify test results. An engineer seeks to understand, validate, and improve the operational context of a system.
- - -Sure, let's go with that for now, but really the difference is an appetite for curiosity. Some people treat monoliths as something to fear or control. Others explore them, learn how to move beyond them, and how to bring others along in the journey.
- - -Testing Is Just a Necessary Tactic of an Engineer
- - -Imagine being an advisor to a professional musician, their performance engineer. What would that involve? You wouldn't just administer tests, you would carefully coach, craft instruction, listen and observe, seek counsel from other musicians and advisors, ultimately to provide the best possible path forward to your client. You would need to know their domain, their processes, their talents and weaknesses, their struggle.
- - -With software teams and complex distributed systems, a lot can go wrong very quickly. Everyone tends to assume their best intentions manifest into their code, that what they build is today's best. Then time goes by and everything more than 6 months old is already brownfield. What if the design of a thing is already so riddled with false assumptions and unknowns that everything is brownfield before it even begins.
- - -Pretend with me for a moment, that if you were to embody the software you write, become your code, and look at your operational lifecycle as if it was your binary career, your future would be a bleak landscape of retirement options. Your code has a half-life.
- - -Everything Is Flawed from the Moment of Inception
- - -Most software is like this...not complete shit but more like well-intentioned gift baskets full of fruits, candies, pretty things, easter eggs, and bunny droppings. Spoils the whole fucking lot when you find them in there. A session management microservice that only starts to lose sessions once a few hundred people are active. An obese 3mb CSS file accidentally included in the final deployment. A reindexing process that tanks your order fulfillment process to 45 seconds, giving customers just enough time to rethink.
- - -Performance engineer doesn't simply polish turds. We help people not to build broken systems to begin with. In planning meetings, we coach people to ask critical performance questions by asking those questions in a way that appeals to their ego and curiosity at a time that's cost effective to do so. We write in BIG BOLD RED SHARPIE in a corner of the sprint board what the percentage slow-down to the login process the nightly build as now caused. We develop an easy way to assess the performance of changes and new code, so that task templates in JIRA can include the "performance checkbox" in a meaningful way with simple steps on a wiki page.
- - -Engineers Ask Questions Because Curiosity Is Their Skill
- - -We ask how a young SRE's good intentions of wrapping u statistical R models from a data sciences product team in Docker containers to speed deployment to production will affect resources, how they intend on measuring the change impact so that the CFO isn't going to be knocking down their door the next day.
- - -We ask why the architects didn't impose requirements on their GraphQL queries to deliver only the fields necessary within JSON responses to mobile app clients, so that developers aren't even allowed to reinvent the 'SELECT * FROM' mistake so rampant in legacy relational and OLAP systems.
- - -We ask what the appropriate limits should be to auto-scaling and load balancing strategies and when we'd like to be alerted that our instance limits and contractual bandwidth limits are approaching cutoff levels. We provide cross-domain expertise from Ops, Dev, and Test to continuously integrate the evidence of false assumptions back into the earliest cycle possible. There should be processes in place to expose and capture things which can't always be known at the time of planning.
- - -Testers ask questions (or should) before they start testing. Entry/exit criteria, requirements gathering, test data, branch coverage expectations, results format, sure. Testing is important but is only a tactic.
- - -Engineers Improve Process, Systems, and Teams
- - -In contrast, engineering has the curiosity and the expertise to get ahead of testing so that when it comes time, the only surprises are the ones that are actually surprising, those problems that no one could have anticipated, and to advise on how to solve them based on evidence and team feedbacks collected throughout planning, implementation, and operation cycles.
- - -An engineer's greatest hope is to make things work really, really well. That hope extends beyond the software, the hardware, and the environment. It includes the teams, the processes, the business risks, and the end-user expectations.
- - -Additional Resources:
- - - - diff --git a/_posts/2019-08-30-on-lack-of-transparency-in-saas-providers.html b/_posts/2019-08-30-on-lack-of-transparency-in-saas-providers.html deleted file mode 100644 index 50968c6..0000000 --- a/_posts/2019-08-30-on-lack-of-transparency-in-saas-providers.html +++ /dev/null @@ -1,110 +0,0 @@ ---- -layout: post -title: On Lack of Transparency in SaaS Providers -date: 2019-08-30 10:45:38.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- DevOps -- performance -tags: [] -meta: - _thumbnail_id: '1246' - dsq_thread_id: '' - _wp_old_slug: on-lack-of-transparency-in-cloud-providers -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2019/08/on-lack-of-transparency-in-saas-providers/" ---- - -As many organizations transition their technical systems to SaaS offerings they don't own or operate, I find it surprising that when a company acquires a 3rd-party offering deployed on said offerings, they are often told to "just trust us" about security, performance, and scalability. I'm a performance nerd, that and DevOps mindset are my most active areas of work and research, so this perspective is scoped to that topic.
- - -In my experience amongst large organizations and DevOps teams, the "hope is not a strategy" principle seems to be missing in the transition from internal team speak and external service agreement. Inside a 3rd-party vendor, say Salesforce Commerce Cloud, I'm sure they very skilled at what they do (I'm not guessing here, I know folks who work in technical teams in Burlington MA). But even espousing a trust-but-verify culture internally, when your statement to customers who are concerned about performance at scale of your offering is "just trust us", seems maligned.
- - -TL;DR: SaaS Providers, Improve Your Transparency
- - -If you provide a shared tenancy service that's based on cloud and I can't acquire service-level performance, security audits, and error logs that are isolated to my account, it's a transparent view into how little your internal processes (if they even exist around these concerns) actually improve service for me, your customer.
- - -If you do provide these metrics to internal [product] teams, ask "why do we do that in the first place?" Consider that the same answers you come up with almost always equally apply to those external consumers who pay for your services that are also technologists, have revenue on the line, and care about delivering value successfully with minimal issues across a continuous delivery model.
- - -If you don't do a good job internally of continuously measuring and synthesizing the importance of performance, security, and error/issue data, please for the love of whatever get on that right now. It helps you, the teams you serve, and ultimately customers to have products and services that are accurate, verifiable, and reliable.
- - -How Do You Move from "Trust Us" to Tangible Outcomes?
- - -Like any good engineer, when a problem is big or ambiguous, start breaking that monolith up. If someone says "trust us", be specific about what you're looking to achieve and what you need to do that, which puts the ownness on them to map what they have to your terms. Sometimes this is easy, other times it's not. Both are useful areas of useful information, what you do know and what you don't. Then you can double-click into how to unpack unknowns (unknowables) in the new landscape.
- - -For SaaS performance, at a high level we look for:
- - --
-
- Uptime and availability reports (general) and the frequency of publication -
- Data on latency, the more granular to service or resource the better -
- Throughput (typically in Mbps etc.) for the domains hosted or serviced -
- Error # and/or rate, and if error detail is also provided in the form of logs -
- Queueing or otherwise service ingress congestion -
- Some gauge or measure of usage vs. [account] limits and capacity -
- Failover and balancing events (such as circuit breaks or load balancing changes) -
You may be hard-pressed to expect some of these pieces of telemetry provided in real-time from your SaaS provider, but they serve as concrete talking points of what typical performance engineering practices need to verify about systems under load.
- - -Real-world Example: Coaching a National Retailer
- - -A message I sent today to a customer, names omitted:
- - -[Dir of Performance Operations],
- - -As I’m on a call with the IEEE on supplier/acquirer semantics in the context of DevOps, it occurs to me that the key element missing in [Retailer's] transition from legacy web solution last year to that which is now deployed via Commerce Cloud, the lack of transparency (or simply not asking on our part) over service underpinnings is a significant risk, both in terms of system readiness and unanticipated costs. My work with the standard brought around two ideas in terms of what [Retailer] should expect from Salesforce:
- - -A) what their process is for verifying the readiness of the services and service-level rendered to [Retailer], and
- - -B) demonstrated evidence of
-what occurs (service levels and failover mechanisms) under significant pressure
-to their services
In the past, [Retailer's] performance engineering practice had the agency to both put pressure on your site/services AND importantly how to measure the impact on your infrastructure. The latter is missing in their service offering, which means that if you run tests and the system results don’t meet your satisfaction, the dialog to resolve them with Salesforce lacks minimum-viable technical discussion points on what is specifically going wrong and how to fix it. This will mean sluggish MTTR and potentially synthesizing the expectation of longer feedback cycles into project/test planning.
- - -Because of shared tenancy, you can’t expect them to hand over server logs, service-level measurements, or real-time entry points to their own internal monitoring solutions. Similarly, no engineering-competent service provider can reasonably expect for consumers to “just trust” that an aggregate product-plus-configuration-plus-customizations solution will perform at large scale, particularly when mission-critical verification was in place before fork-lifting your digital front door to Salesforce. We [vendor] see this need for independent verification of COTS all the time across many industries, despite a lack of proof of failure in the past.
- - -My recommendation is that, as
-a goal of what you started by creating a ticket with them on this topic, we
-should progressively seek to receive thorough information on points A and B
-above from a product-level authority (i.e. product team). If that’s via a
-support or account rep, that’s fine, but it should be adequate for you to be
-able to ask more informed questions about architectural service limits, balancing,
-and failover.
//Paul
- - -What Do You Think?
- - -I'm always seeking other perspectives that my own. If you have a story to tell, question, or otherwise augmentation to this post, please do leave a comment. You can also reach out to me on Twitter, LinkedIn, or email ["me" -at-- "paulsbruce" --dot- "io"]. My typical SLA for latency is less than 48hrs unless requests are malformed or malicious.
- diff --git a/_posts/2019-09-15-this-is-why-devops.html b/_posts/2019-09-15-this-is-why-devops.html deleted file mode 100644 index e62ed50..0000000 --- a/_posts/2019-09-15-this-is-why-devops.html +++ /dev/null @@ -1,136 +0,0 @@ ---- -layout: post -title: 'This is Why #DevOps' -date: 2019-09-15 16:39:32.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- advocacy -- DevOps -- quality -tags: [] -meta: - _thumbnail_id: '1254' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2019/09/this-is-why-devops/" ---- - -Since well before 2008, DevOps as a keyword has been growing steadily in mindshare. This post is my version of a landing page for conversations I have with folks who either know little about it, know a lot about it, or simply want to learn why DevOps is not a hashtag buzzword.
- - -TL;DR: if you're not interested in reading a few paragraphs of text, then nevermind. If you really must jump to my own articulation of DevOps, click here.
- - -DevOps Is a Question, Not an Answer
- - -The word "DevOps" is a starting point in my conversations with folks along the journey. Whether it's over obligatory beers with local community members after a monthly meetup, in a high-pressure F100 conference room, in my weekly meetings with the IEEE, or internally in organizations I work with and accept benefits from, "DevOps" is often the start of a really important dialog. It's a question more than a statement, at least the way I use it day to day, spiked with a small challenge.
- - -It's a question of "what are you doing now, and where do you want to be". It's a question of "how are the things you're doing now incrementally improving, maybe compounding, your efforts". It's a question about how you remove both your backlog of toil and resolve the inbound toil that daily changes to software systems represents. It's a question of "how do you chose to think about your work as you're doing it?" DevOps is an improvement mindset, not just about code and configuration, but about organizational cohesion, communication, collaboration, and engineering culture.
- - -DevOps Is More than "Developers and Operations"
- - -Unlike those whose staunch opinions limit the scope of DevOps to refer to activities of developers and operations engineers, though managing scope creep is important, the agency and effect of DevOps are clearly intertwined with an organizations capability to also foster a set of core values and principles. For over a decade, we've seen some teams going so much faster than others that it doesn't matter who's slower, the result out to production is buggy, insecure, and unscaleable systems. This very common and dysfunctional clock speed mismatch also extends to testing, risk and compliance, budgeting (i.e. part of planning), architecture, and proper monitoring/telemetry/measurement practices.
- - -Similarly, an 'agile' team find it very hard to be 'agile' without the buy-in of leadership, support from finance and legal, alignment with marketing and sales, and deep connection to its customers/stakeholders as well as a whole slew of other examples that underline how agile can't be just a dev-centric worldview.
- - -I hesitate to invent more terms such as 'DevBizOps', 'DevTestOps', and 'DevSecOps' simply to segment off these augmented versions of DevOps when these areas of concern should be integrated into systems delivery lifecycles, to begin with. I don't want to conflate concepts, yet so many elements overlap, it's hard to know how to have one conversation without having three others too. At the end of the day though, it's as arbitrary to me that people invent new terminology as it is that some others chose to dig their fundamentalist heels in about exclusively keeping the scope of DevOps to two traditionally distinct groups, particularly when we see modern interpretations of DevOps blends skills and responsibilities to the point that we highly value (pay) SREs and "10x developers" because they defy narrowly scoped job opportunities in favor of cross-functional roles.
- - -What are *My* Core Values and Principles of DevOps?
- - -Again, this is a question you should seek to answer in your own way, but after only five years of immersion and purposefully seeking out other perspectives daily, a few key elements seem to hold:
- - --
-
- Transparency
-
-
-
- Observability -
- Traceability -
- Open Systems -
- Great Documentation -
- - Inclusivity
-
-
-
- Collaboration -
- Swarming / Blameless Fault Handling -
- Customer/User-focus -
- Prioritization to Value -
- - Measurement
-
-
-
- Building Quality In -
- High-value Feedback Loops -
- Useful Monitoring and Telemetry -
- Clear SLA/SLO/SLIs -
- - Improvement
-
-
-
- Automate AMAP and by default -
- Continuous Learning -
- Coaching and Mentoring -
- - Alignment
-
-
-
- Small Batches, Short-lived Change Cycles -
- Clear Processes (Onboarding, Approvals, Operationalizing, Patching, Disposal) -
- Work Contextualized to Biz Value/Risk -
- All Stakeholders Represented in Decisions -
-
DevOps Is Inclusive by Nature
- - -When dialed in, the values and principles of DevOps foster an environment of high-trust and safety, synthesize perspectives without losing focus, and balance personal and team capacity to compound individual contributions rather than burn out talented professionals. DevOps is not, as Audre Lord (via Kelsey Merkley) puts it, our "master's tool", so long as we don't make it such; rather, if we decide that DevOps must not inherit the meritocracy and exclusivity of corporate management and Agilista methodology, it must be a truly better alternative than that which came before.
- - -This is how DevOps can change things, this is why it must. New thinking and new voices provide a way out of tech monoculture. In order to improve a system, you need more than the one option you've already found faulty. This is why I personally support local non-profits like Resilient Coders of Boston. DevOps values and principles offer a far better chance for new voices to be at the table, at least the version I'm advocating.
- - -DevOps Improves Organizations In-Place
- - -Imagine if your teams, your colleagues, your leadership really internalized and implemented the core values and principles defined above. Doing so in finance, human resources, marketing, sales, and even [dare I say] board of trustees groups would cause a natural affinity for supporting systems engineering teams. Conversely too, developers about to change a line of code would be more likely to ask "how does this impact the customer and my organization's goals?", "what documentation do customers and sales engineers need to effectively amplify the new feature I'm developing?", and "how could this process be improved for everyone?".
- - -There is an old stereotype of programmers being bad at "soft skills", resulting in miscommunication, disconnect of work from business value, "not my problem" mentality, and throw-it-over-the-fence mindset. My perspective on DevOps is that none of these things would have the room to materialize in organizations that put processes in place to ensure the above values and principals are the norm.
- - -Everyone can do these things, there's nothing stopping us, unicorn to F100. Importantly, transitioning from what is currently in place to these values takes time. Since time is money, it takes money too. DevOps advocacy isn't good enough to make the case for these changes and the spend that comes attached. No one developer or even team can change an organization, it takes Gestalt thinking and demonstrated value to get people 'bought in' to change.
- - -Facilitating DevOps Buy-In
- - -The first thing to get straight about DevOps is that it takes conversation and active engagement. This is not and never should be something you go to school or get a certificate for; it is a journey more akin to learning a martial art like Uechi-ryu than a course you pay thousands of dollars for at some tech industry Carnivale.
- - -Collect those in your organization is interested in having a conversation about cultural implications of DevOps values and principles, using something lightweight like a lunchtime guild, a book club, or even a Slack channel. Listen to people, what they think and already know, and don't assume your perspective is more accurate than theirs. Be consistent about when/what you do together to further the dialog, hold a local open spaces meetup (or find someone who does this, like me), and invite people from outside the engineering team scope, such as a VP or member of product management at your organization, and ASK THEM afterwards what they thought.
- - -Once you have people from different levels engaging on similar wavelengths about DevOps, ask them to help each other understand what are some first tangible and tactical steps to improve the current situation based on some of the core values and principles either defined above or further crafted to fit your circumstances. Get a headline name in one or more regional DevOpsDays organizing committees to come visit as an outside perspective to what you've got going. And importantly, make time for this improvement. SEV1 incidents aside, there's always some weekly space on everyone's calendar that's an optimal time to get together.
- - -Or you can just ping me [at] paulsbruce [dot] io or on Twitter and we can figure out a good way for you to get started on your own DevOps journey. I'm always happy to help.
- diff --git a/_posts/2019-10-19-on-volunteering-for-tech-community-work.html b/_posts/2019-10-19-on-volunteering-for-tech-community-work.html deleted file mode 100644 index e5429ed..0000000 --- a/_posts/2019-10-19-on-volunteering-for-tech-community-work.html +++ /dev/null @@ -1,154 +0,0 @@ ---- -layout: post -title: On Volunteering for Tech Community Work -date: 2019-10-19 15:49:39.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: [] -tags: [] -meta: - _thumbnail_id: '1266' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2019/10/on-volunteering-for-tech-community-work/" ---- - -This is my attempt to distill what I've learned over the past two years of contributing to maintaining and improving the local DevOps community in Boston. As in all cases, what we think we "know" at a point in time can/may/should change as our journey progresses. Rather than prescriptive, this article is descriptive of my current approach.
- - -TL;DR
- - --
-
- do it for 'right' reasons that align with community values -
- really know what you're getting into and commit proportionally -
- be prepared to spend time and energy, particularly on personal adaptation -
- manage your life work-in-process limit carefully -
- expect something unexpected out of it -
A True Community is Tangible
- - -There are many definitions of "community", but here's mine at this point:
- - -- - -A group of people dedicated to sharing and improving along with each other.
-- me, provisionally as of 2019
If helping to do this sounds easy, it is not. Dedication means graciously and humbly collaborating with other organizers, diligently following up with a million little details, listening and understanding where people in the community are coming from, actively seeking personal improvement and synthesizing into volunteer work, responding quickly to important issues in the community and of course, making/taking time from your personal and professional life to do these things reliably.
- - -For an organizer, these things are very tangible because they cost us what we can never get back: time. For other community members, the value comes from elements like knowledge sharing, conversations, job opportunities, or even a sense of simply being part of a community. Showing up at a meetup event or a casual coffee chat is the first step, and understanding why you enjoyed brings you back.
- - -One of the most concrete, tangible outcomes of being part of the community I work in is that there's always an opportunity to learn something new and useful, which by definition you can't predict. The more you give, the more you get. The more you're open-minded, the more mindful and receptive to good ideas you become. The more you listen, the more you hear.
- - -Fostering Unity and Diversity at the Same Time
- - -We all come to the table with a variety of backgrounds, experiences, and perspectives. This is the true strength of a healthy community. Balancing time constraints and the distribution curve of appetites for themes is tricky, but doable when facilitators are dedicated and have the interests of each other (including the whole community) in mind.
- - -There's usually a unifying set of core topics and themes related to some common interests (such as learning new tech or how to deal with office culture or professional improvements). Straying too far outside this focus often contributes to our "variety" getting in the way of sharing and improving, though sometimes including topics seemingly unrelated to the tech side of the conversation can provide opportunities for divergent thinking and new knowledge.
- - -An Organizer's Roles and Responsibilities in the Community
- - -At a small scale, this may happen organically, but as the group scales, this doesn't happen without organizers. A community organizer is someone who:
- - --
-
- looks out for the best interests of all members of the community -
- is dedicated to creating more value than they capture [1] -
- ensures that underrepresented perspectives have a voice -
- deals with the logistics of shared spaces (events, chats) -
- ensures that the values and principles in the Code of Conduct are respected -
- develops a plan of succession and redundancy in organizing responsibilities -
- insulates the community from negative or unwanted solicitations -
I've had to great privilege to work with a bunch of other organizers and volunteers, and one of the key things I see make up for the aggregation of individual intents and sometimes dysfunctions is the willingness to come back and do/be better.
- - -One thing that's always a challenge is to maintain a pool of organizers greater than one or two active individuals. Think of it as resiliency engineering, a topic close to my personal wheelhouse. The more trustworthy and reliable individuals are involved, the more trustworthy and reliable the outcome.
- - -Thrash vs. Progress
- - -Everyone brings their own personal interests into a group equation. When the guiding principles include alignment and responsibility, it works really well for everyone. When someone puts their personal interests above the common good, all is not lost, but causes a lot of "thrash". I define thrash as work which ultimately does not progress and improve the common good.
- - -Some folks need to thrash around with new ideas and values, and this is okay too. Healthy synthesis of new ideas and values is important. Provided that there's room for people to bring these ideas to the table and that progress on existing values is in play, we all win.
- - -Getting Ahead of Toxicisity
- - -I've worked in a number of places and communities where behavior is sub-par. When something is introduced which causes a body (or community) to make a backward motion towards it's shared and the common good, particularly on a repeated basis, I would call this "toxic". In a DevOps mindset, this can happen too, where anti-patterns such as burnout and distrust crop up because little details aren't being addressed.
- - -In DevOps communities, many are familiar with the Westrum model of organizational culture: pathological, bureaucratic, and generative. Though often misused to label groups and people at an aggregate level, the most useful employment of this lens IMO is around behaviors and activities. Those can be changed, those can be provided guidance where people have specific decisions to make.
- - -Two examples from our local meetup come to mind: use of alcohol and meritocratic rhetoric. Though innocuous in small amounts, both quickly detract from community spirit and sense of safety in our culture.
- - -In one thread, someone suggested that we bring our own bottles of scotch. I love my happy hours the same as warm-blood, but immediately before that, I witnessed and heard from a number of other community members that an unannounced and unplanned open bar at a meetup detracted from fostering a safe space. I quickly hopped on this, making sure that we all consider how this idea contributes to values in our Code of Conduct. Call me the culture cop, but after collaborating with other organizers on the approach, it was clear that we wanted to get ahead of where it was going and pivot the conversation back to the purpose of the community.
- - -In another backchannel thread with community members, we discussed the rising trend of conversations assuming that everyone had the same opportunities leading to a notion that all people would be rewarded based on the merit of their works. In the current tech culture, meritocracy is rampant, most notably due to open-source mainstays such as the Linux Code of Conduct and revulsion of GNU founder R.M.S after a career of misogynistic and belligerent behavior. Not everyone can just walk up to a table and be valued by those there. As a common good of our community, we do our best to steer away from the 'meritocracy lens' and bring conversations back to a place that is useful for all members of the community.
- - -Improvement Means Earning and Granting Trust
- - -One of my biggest goals is to find someone better than me at these things that not only fulfills above bulleted organizer requirements but that I can trust to improve and grow the community in ways I can't. Knowing that I probably won't be able to keep up the same level of commitment to our community indefinitely, succeeding myself with other trustworthy individuals is not a zero-effort endeavor in the slightest. Nothing happens overnight, and the best time to do something truly good is always 'now'.
- - -It's definitely been interesting to co-organize the meetup, facilitating events and threads, hopefully earning an amount of trust in our community about why I'm spending my time and energies on it. What I've learned is that granting someone trust is a very different proposition than earning it. One thing suggested to me in a backchannel was that "ally is a term someone uses on you, not something you ". After bouncing this around in the back of my mind for months, I've realized that I don't need to focus on wait for people to ascribe community members (especially organizers) trust, we just need to do the best we can and demonstrate trustworthy behavior.
- - -As one of 6 organizers of the yearly DevOpsDays Boston event, I was in a place to understand that forging an initial relationship with the Inclusive Tech Lab and Resilient Coders communities was a great direction (most props go to Laura Stone, another co-organizer of the meetup and event). The heart and spirit of these other groups align with the values and demonstrated levels of commitment that I know are required to drive the community forward.
- - -Listening to and Integrating Feedback
- - -Imagine all the times that someone spoke words that you didn't understand...thing you weren't in a place to receive. What a shame. What do you do about that?
- - -Mostly always looking to get better, but one thing I've realized is that the moments my mouth is open should be marginal compared to time I spend hearing what others have to share. If doing it right, the times I speak should be guided by three questions:
- - --
-
- does this have to be said (and why)? -
- does this have to be said by me? -
- does this have to be said right now? -
Though everyone suffers from subjectivity (implicit in the above bullets), listening and empathizing with what you know about someone's perspective increases the "you AND them" funnel for new and important ideas. Also, reading or listening to lots of books and blogs also widens perspective.
- - -Get Involved, Be Good
- - -Though not a requirement to attending, I'm surprised at the number of regular attendees who haven't actually read the Code of Conduct but still naturally adhere to the values and principles within. I take this as a sign of common interest in the community but don't take for granted the responsibility to make sure behaviors are inline.
- - -If you're thinking of helping, definitely reach out. If not, still, leave comments. I appreciate feedback and engagement on approach to facilitating in the community, always.
- diff --git a/_posts/2019-10-23-crossing-cross-functional-chasms.html b/_posts/2019-10-23-crossing-cross-functional-chasms.html deleted file mode 100644 index b4e9365..0000000 --- a/_posts/2019-10-23-crossing-cross-functional-chasms.html +++ /dev/null @@ -1,95 +0,0 @@ ---- -layout: post -title: Crossing Cross-functional Chasms -date: 2019-10-23 13:13:09.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- DevOps -- teams -tags: [] -meta: - _thumbnail_id: '1270' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2019/10/crossing-cross-functional-chasms/" ---- - -Initialization Phase
- - -My first evening in La Ciotat: I picked up a rental car in town due to the good graces of Giulia, the front desk assistant who was coming off of her shift. She called, then insisted she drive me to the pick-up office where the lady attending was on her way out to deliver a car. Thirty not so awkward minutes later after discussing dog grooming and training techniques in great depth, the attendant came back and shortly I had a car. It was just starting to rain and it had been years since my last stick shift. Crossed fingers and no stalls to get back to La Rose, but by that time, waters were pouring out in sheets. The sprint from car to lobby was in place of the shower I had hoped to take earlier.
- - -The hotel restaurant was leaking everywhere and occasionally losing power. The great thing about room charges is that a full bar downstairs doesn't require the internet to hand over all sorts of drinks. People outside the lower forty-eight seem to intuit what to do when the credit card terminal is out of service. The lightning was faster and closer than I've ever seen from my fishing town, so it was a good time to revert to battery power and write this.
- - -My recent recipe is bourbon (or tequila) and a splash of each lemon juice, creme de cacao, absinthe, shaken and filtered into a highball with a thick peel of an orange. A few months ago it was half high-quality sake and half Prosecco with a flake of rosemary. In a pinch, anything works, and when your bartender has got flooding issues to deal with, you can ponder life under a canopy and try to stay dry. The following are my thoughts from underneath all of this.
- - -Planning Phase
- - -The thing about my work, it isn't scalable because it serves different goals than other kinds of work. Like Kent Beck describes in his "3-x" model, there are modes of work that optimize for different localized outcomes but all serve the same high-order goal. What is that goal? As Eliyahu Goldratt states, "the goal of an organization is to make money". Certainly commercial ones, but even non-profits need to do this in order to exist. I exercise aspects of each of Kent's 3 modes: explore, expand, and extract. I dig holes to find gold and when I hit it I dig hard, and then try to scale that out to optimize efforts to extract that gold.
- - -In his epic distillations, the Innovators Dilemma and Solution, Clayton Christensen puts a fine point on how if a company is not thinking of its next horizon at all points in the current extract motion, it has no lasting future. Despite the dilemma of where to divest funds and how to prioritize "next" work, I am looking to do that for whomever I work with. I want to help optimize what's currently being extracted, translate learnings into gaps and undiscovered opportunities, and continuously listen and learn "what's next" (ref: Tim O'Reilly in "WFT?: What's the Future and Why It's Up to Us". If we're not doing that, either homogeneously as all actors in an organization or as a unique sub-function, then we're dooming our employees and product to obsolescence.
- - -Implementation Phase
- - -I do…a lot…of things in my current organization. Pre-sales guidance, analyst relations, strategic product planning, blog writing, speaking, webinars, on-site customer planning sessions, technical prototyping, automated pipeline and testing examples, collaboration with support on key customers, building examples, positioning and messaging assistance, customer engineering, amongst others. "Cross-functional" is an easy way of putting it. When friends ask about what I do, I just say "I help technology companies make better decisions."
- - -But when your cross-functional, you get to see how diverse people groups are and how differently they structure their goals. For some, it's money, but for others it's lead acquisition, and for yet others, it's sprint deliverables and low-defect release cycles. For leadership it's all of these things plus happy employees, happy customers, and happy selves. I want all of these things and more…happy me and happy mine (family) which requires balance. Balancing multiple objectives takes a lot of practice, similar to my experiences with Uechi Karate-do. Balance isn't a static state, it's an ability to re-balance and prioritize based on shifting needs and constraints.
- - -In planning one of four strategy sessions with one of the founders, I found myself thinking "our goals are not the same, he wants to prioritize an existing backlog around reporting, but I want to define the new state of the art for our industry". After realizing that he had a different goal, we played better together; but I am not distracted. Maintaining the status quo has never been my strong suit and I'm more useful when focused on what's next, not just what already was.
- - -This is my current approach to balance: understand what drives people (myself included), listen to everyone, provide value that's aligned to these motives, and circulate what makes sense to the organization. Catching the moment when a founder's goals and my own differ happens in real-time, but only if I'm exercising balance along these guidelines.
- - -Deployment Phase
- - -This week, the plan is to listen, a lot. Especially because of the language gap, but also thanks to my eclectic manners of verbal communication, as evidenced last time, less seems to be more here. I am working to lock down the details of a new position, focused on bringing the customer perspective to every area of business and a translation of my own invention. The activities performed today have a tendency to be…predictable, easily replicable, and therefore boring to someone like me.
- - -Though billed as "strategy sessions", my feeling is that current leadership understands the need for all elements of the business to be engaged deeply…"lean forward" as I often call it. The real strategy happens next week, in decision meetings amongst founders and key business owners separate from the rest of the employees. This is an interesting model, right-fit I think for humans who need time to digest and consider various perspectives and potential directions.
- - -Though many ideas and directions will be discussed over the next 7 days, we'll need to prioritize and I don't own the company. All I can do is help those who do have a clear understanding of internal and external dynamics, provide requisite evidence for my positions, and improve relationships with my counterparts here in France.
- - -Monitoring Phase
- - -Feedback loops are important whether you automate them or not (but automating them is the smart way to do it). How do you automate human interactions though? The closest I've gotten is to "pull forward"…in my upcoming role, building in the demand and supply of effective internal and external collaboration. The partner channel is a significant dynamic in my current organization, much of it is channel but there is also a contingent technical element, as all good partnerships between tech companies should have. A colleague of ming is fantastic at tactical and technical delivery in this scope, but to scale these efforts out to the whole organization takes project/program management that he's not particularly keen to deliver himself.
- - -A key element to "monitoring" my effect is to A) have traceable inclusion in conversations (via Salesforce currently) and B) through volunteered backchannel context measure how many times my involvement improves what we're doing, in the partner, sales, marketing, and development work. This week would be an example of execution, then after next week, I would explicitly ask my leadership what value they heard as a result of my presence. Pull forward isn't a zero-effort enterprise, but is absolutely necessary if you take your individual impact lifecycle seriously.
- - -Reintegration Phase
- - -The bartender tonight quickly called for backup and started taking care of the flooding issues. I suspect this was because he knew that if he had just sat there and let the hotel restaurant get flooded, someone would ream the shit out of him the next day. He got ahead of the problem and solved it. This is what DevOps and SRE is all about, seeing that no one else is solving for the lasting value and putting patterns in place to help others to do exactly that.
- - -In our current state, this organization takes its time to synthesize and integrate learnings. Faster than most other teams I see but not as fast as some, as with everything pace can be improved. More importantly, alignment and transparency must always be improved and that is not zero-effort in the slightest either. In a prior position, an SVP once stated that "alignment is about 50% of my time spend". With marginal variance, I wish that applied across all roles and responsibilities in every team and every organization I work with. Imagine the impact you'd have if 100% of your 50% heads-down work was wildly effective. That is what rowing in the same direction looks like.
- - -Reprise
- - -For this post, there is no "call to action" other than if you want to leave a comment or engage on social channels. I can always find something worthwhile for you and I to do together. Drop a line and let's chat about that.
- diff --git a/_posts/2019-10-27-afterthoughts-on-hive-minding.html b/_posts/2019-10-27-afterthoughts-on-hive-minding.html deleted file mode 100644 index 877cf92..0000000 --- a/_posts/2019-10-27-afterthoughts-on-hive-minding.html +++ /dev/null @@ -1,147 +0,0 @@ ---- -layout: post -title: Afterthoughts on Hive Minding -date: 2019-10-27 08:39:40.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- advocacy -- DevOps -- management theory -- performance -- social -- teams -- Zen-in -tags: [] -meta: - _thumbnail_id: '1285' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2019/10/afterthoughts-on-hive-minding/" ---- - -It's a powerful thing to understand how your brain works, what motivates you, and what you don't care about. There are so many things that can distract, but at the end of the day, there are very few things measurable immediately worth having done. Shipping myself to Europe until next week, for example, has already had measurable personal and professional impact.
- - -One thing I experienced this week after injecting a little disruption to conformity yesterday was what I now call "hive minding", or otherwise assisting independent contributors in rowing in the same direction. The classical stereotype of "herding cats" infers that actors only care about themselves, but unlike cats, a bee colony shares an intuitive, survival imperative to build and improve the structure that ensures their survival. Each bee might not consciously think about "lasting value", but it's built into their nature.
- - -Be Kind, Rewind
- - -I'm always restless, every success followed by a new challenge, and I wouldn't have it any other way, but it does lead to a growing consideration about plateauing. Plateauing is a million times worse than burning out. There are plenty of people and companies that have burned out already but are still doing something "functional" in a dysfunctional industry, and if the decision is to flip that investment, it's an easy one to make. Fire them, trade or cut funding; but what do you do with a resource when they plateau?
- - -I think you'll know you've plateaued when you find yourself without restlessness. If necessity is the mother of invention, restlessness is the chambermaid of clean mind. Al least for me, like a hungry tiger in a cave, I must feed my restlessness with purposeful and aligned professional work. The only problematic moment with me...I like to get ahead of the problem of someone telling me what to do by figuring out what we (everyone, me and them) should be doing before someone dictates it with less context.
- - -The sweet spot of this motion is to do this together, not in isolation and not dictatorially, but coalescing the importance of arriving at the "right" goals and in alignment at the same time. The only surprises when you're riding the wave together is what comes next, and when you engineer this into the process, surprises are mostly good.
- - -It took a while to arrive at this position. I had to roll up sleeves, work with many different teams in multiple organizations, listen to those whose shoes I don't have the time or aptitude to fill, figure out how to synthesize their inputs into cogent and agreeable outcomes, and do so with a level of continuity that distinguishes this approach from traditional forms of management and group facilitation.
- - -Don't Try This On Your Own
- - -The cost of adaptability is very high. If I didn't have an equally dedicated partner to run the homefront, none of this would work. She's sought out the same kind of commitment and focus on raising the kids as I do with what goes into pays the bills. There are very few character traits and creature comforts we share, but in our obsession over the things that make the absolute best come out of what we have, she more than completes the situation.
- - -In this lifestyle, I have to determine day by day and week by week what net-new motions/motivations I need to pick up and which I need to put down, either temporarily or permanently. This can feel like thrash to some, but for me, every day is a chance to re-assess based on all the days before now; I can either take that opportunity or not, but it is there despite whether I do or not take it. If my decisions are only made in big batches, similar to code/product releases, I inherit the complexities and inefficiencies of "big measurement"...namely, granularity in iterative improvement.
- - -Feedback Loops, Everywhere
- - -As I explore the dynamics of continuous feedback loops beyond software and into human systems, a model of frequency in feedback and software delivery not as separate mechanisms, but as symbiotic, emerges. The more frequently you release, the more chances there are for feedback. The more feedback you can synthesize into value, the more frequently you want to release. One does not 'predict' the other; their rate bounds each other, like a non-binary statistical model.
- - -What I mean is that a slow-release cycle predicts slow feedback and slow feedback predicts low value from releasing frequently; a fast feedback mechanism addicts people to faster release cycles. They share the relationship and depending on how extreme the dynamics feeding into one side of the relationship, the other one suffers. Maybe at some point, it's a lost cause.
- - -An example from the performance and reliability wheelhouse is low/slow performance observability. When you can't see what's causing a severe production incident, the live investigation and post-mortem activity is slow and takes time away from engineering a more reliable solution. Firefighting takes dev, SRE, ops, and product management time...it's just a fact. Teams that understand the underlying relationship and synthesize that back into their work tend to use SEV1 incidents as teachable moments to improve visibility on underlying systems AND behavioral predictors (critical system queue lengths, what levels of capacity use constitute "before critical", architectural bottlenecks that inform priorities on reducing "tech debt", etc.).
- - -The point is that feedback loops take time and iterative learning to properly inject in a way that has a positive, measurable impact on product delivery and team dynamics.
- - -Going from Feedback Loops to Iterations...Together
- - -All effect feedback loops have one thing in common: they measure achievement levels framed by a shared goal. So you really have to work to uncovered shared goals in a team. If they suit you and/or if you can accept the awesome responsibility to challenge and change them over time, it's a wild ride of learning and transforming. If not, find another team, company, or tribe. Everyone needs a mountain they can traverse and shouldn't put themselves up to a trail that will destroy them. This is why occasionally stepping back, collaborating, and reporting out what works and what doesn't is so important. Re-enter the concept of "team meetings".
- - -Increasingly, most engineers I talk to abhor the notion of more meetings, usually because they've experienced their fair share of meetings that don't respect their time or where their inputs have not been respectfully synthesized in a way they can see. So what, meetings are a bad thing?
- - -Well, no, not if your meetings are very well run. This is not one person's job, though scrumbags and mid-level management with confirmation bias abound, and especially so because they don't have built-in NPS (net promoter score). A solution I've seen to the anti-pattern of ineffective meetings is to establish common knowledge of what/how/why an "effective" meeting looks like and expect these behaviors from everyone in on the team and in the org.
- - -How to Encourage Effective Collaboration in Meetings
- - -Learn to listen, synthesize, and articulate back in real-time. Too much time goes by, delay and context evaporate like winter breath. Capture as much of this context as you can while respecting the flow of the conversation. This will help you and others with remembering and respecting the "why", and will allow people to see what was missing (perspectives, thinking, constructs), afterward. Examples of capture include meeting minutes, pictures of post-its, non-private notes from everyone, and even recordings.
- - -But in just about every team and organization there's a rampant misconception that ALL meetings must produce outcomes that look like decisions or action items. These are very beneficial, but I've seen people become anti-productive when treating themselves and others as slaves to these outcomes. Taking decisions too early drives convergent attitudes that are often uninformed, under-aligned, and often destructive.
- - -Some of the most effective meetings I've had share the following patterns:
- - --
-
- know why you're meeting, provide context before, and set realistic expectations -
- have the "right" people in the room
-
-
-
- who benefit from the anticipated outcomes and therefore are invested in them -
- who bring absolutely critical perspective, where otherwise invalidates outcomes or cause significant toil to refactor back in afterward; not to few -
- who contribute to functional outcome (as opposed to those who are known to bring dysfunction, don't respect the time of others, argue over align); too many -
- - agree on what positive and negative outcomes look like before starting in -
- use communication constructs to keep people on track with producing outcomes -
- have someone to ensure (not necessarily do all the) capture; note and picture taker -
- outcomes are categorized as:
-
-
-
- clear, aligned decisions (what will happen, what worked, what didn't, what next) -
- concrete concerns and missing inputs that represent blockers to the above -
- themes and sense of directional changes (i.e. we think we need to change X) -
- all info captured and provided as additional context for others -
-
Trust AND Verify
- - -One thing I keep finding useful is to challenge the "but" in "trust, but verify". In English, the word "but" carries a negating connotation. It invalidates all that was said before it. "Your input was super important, BUT it's hard to understand how it's useful"...basically means "Your input was not important because it was not usable."
- - -My alternative is to "trust and verify", but with a twist. If you're doing it right, trust is easy if you preemptively provided an easy means to verify it. If you provide evidence along with your opinion, reasonable people are likely to trust your judgment. For me, rolling up the sleeves is a very important tool in my toolbelt to produce evidence for or against a particular position. I know there are other methods, both legitimate and nefarious, but I find that practical experience is far more defensible than constructing decisions based on shaky foundations.
- - -All this said, even if you're delivering self-evident verification with your work, people relationships take time and certainly take more than one or two demonstrative examples of trustability to attain a momentum of their own. Trust takes time, is all.
- - -Takeaways and Action Items from This Week
- - -Democratic decision processes are "thrashy". Laws and sausages: no one wants to know how they're made. In small teams going fast, we don't have the luxury of being ignorant of outcomes and the context behind them. For some people, "democracy" feels better than dictatorial decisions being handed down without context; but for those who still find a way to complain about the outcomes, they need to ask themselves, "did I really care enough to engage in a functional and useful way, and did I even bother to educate myself on the context behind the decision I don't like?"
- - -Just like missing a critical perspective in a software team, in a global organization, when one region or office dominates an area of business (U.S. on sales, EU on security, for instance), this will inevitably bias outcomes and decisions affecting everyone. As the individual that I report to puts it, "scalability matters to every idea, not just when we're ready to deploy that idea". Make sure you have the right "everyone" in the room, depending on the context of your work and organizational culture.
- - -Someone I once met and deeply respect once told me "it's not enough to be an ally, you need to be an accomplice". In context, she was referring to improving the epic dysfunction of modern technology culture by purposefully including underrepresented persons. Even if we make a 10% improvement to women's salaries, hire more African-American engineers, create a safer place for LGBTQ, I still agree with the premise that doing these things isn't good enough. Put it another way, receiving critical medical treatment for a gushing head wound isn't an "over-compensation", it's a measured response to the situation. The technology gushing head wound, in this case, is an almost complete denial from WGLM (white guys like me) that there is a problem, that doing nothing continuously enables the causes of the problem, that leadership on this doesn't necessarily look or think like us, and that this isn't necessarily needed now.
- - -Bringing it back to the wheelhouse of this article, true improvement culture doesn't just take saying "sure, let me wave at you as an ally while you go improve the team". It takes being an accomplice (think a getaway driver), we should ALL be complicit in decisions and improvement. Put some skin in the game, figure out how something truly worth improvement and your effort maps to your current WiP (work in progress) limits, and you may find that you need to put something less worth your time down before you can effectively contribute to improvement work. Surrounding yourself with folks who get this too will also increase the chances that you'll all succeed. This is not an over-compensation, it is what everyone needs to do now to thrive, not just survive.
- diff --git a/_posts/2019-12-14-put-that-in-your-pipeline-and-smoke-test-it.html b/_posts/2019-12-14-put-that-in-your-pipeline-and-smoke-test-it.html deleted file mode 100644 index 2a51526..0000000 --- a/_posts/2019-12-14-put-that-in-your-pipeline-and-smoke-test-it.html +++ /dev/null @@ -1,249 +0,0 @@ ---- -layout: post -title: Put That in Your Pipeline and Smoke Test It! -date: 2019-12-14 13:18:33.000000000 -05:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- automation -- developer -- DevOps -- performance -- quality -- software -- testing -tags: [] -meta: - _oembed_332217f052ff05ae16069b5d8f0bb214: - _oembed_time_332217f052ff05ae16069b5d8f0bb214: '1576500610' - _thumbnail_id: '1314' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2019/12/put-that-in-your-pipeline-and-smoke-test-it/" ---- - -I rarely bother to open my mouth as a speaker and step into a spotlight anymore. I've been mostly focused on observing, listening, and organizing tech communities in my local Boston area for the past few years. I just find that others'
- - -A friend of mine asked if I would present at the local Ministry of Testing meetup, and since she did me a huge last-minute favor last month, I was more than happy to oblige.
- - -"Testing Is Always Interesting Enough to Blog About"
- - -
-
The state and craft of quality (not to mention performance) engineering has changed dramatically in the past 5 years since I purposely committed to it. After wasting most of my early tech career as a developer not writing testable software, the latter part of my career as of late has been what some might consider penance to that effect.
- - -I now work in the reliability engineering space. More specifically, I'm a Director of Customer Engineering at a company focusing on the F500. As a performance nerd, everything inherits a statistical perspective, not excluding how I view people, process, and technology. In this demographic, "maturity" models are a complex curve across dozens of teams and a history of IT decisions, not something you can pull out of an Agilista's sardine can or teach like the CMMI once thought it could.
- - -A Presentation as Aperitif to Hive Minding
- - -This presentation is a distillation of those experiences to date as research and mostly inspired to learn what other practitioners like me think when faced with challenges in translating the importance of holistic thinking around software quality to business leaders.
- - -Slides: bit.ly/put-that-in-your-pipeline-2019
- - -Like I say at the beginning of this presentation, the goal is to incite collaboration about concepts, sharing the puzzle pieces I am actively working to clarify so that the whole group can get involved with each other in a constructive manner.
- - -Hive Minding on What Can/Must/Shouldn't Be Tested
- - -The phrase 'Hive Minding' is (to my knowledge and Google results) a turn-of-phrase invention of my own. It's one incremental iteration past my work and research in open spaces, emphasizing the notions of:
- - --
-
- Collective, aggregated collaboration -
- Striking a balance between personal and real-time thinking -
- Mindful, structured interactions to optimize outcomes -
At this meetup, I beta launched the 1-2-4-All method from Liberating Structures that seemed to work so well when I was in France at a product strategy session last month. It so well balanced the opposite divergent and convergent modes of thinking, as discussed in The Creative Thinker's Toolkit, that I was compelled again to continue my active research into improving group facilitation.
- - -Even after a few people had to leave the meetup early, there were still six groups of four. In France there were eight contributors, so I felt that this time I had a manageable but still scaled (4x) experiment of how hive minding works with larger groups.
- - -My personal key learnings
- - -Before I share some of the community feedbacks (below), I should mention what I as the organizer saw during and as outcomes after the meetup:
- - --
-
- I need to use a bell or chime sound on my phone rather than having to interrupt people once the timers elapse for each of the 1-2-4 sessions; I hate stopping good conversation just because there's a pre-agreed-to meeting structure. -
- We were able to expose non-quality-engineer people (like SysOps and managers) to concepts new to them, such as negative testing and service virtualization; hopefully next time they're hiring a QA manager, they'll have new things to chat about -
- Many people confirmed some of the hypotheses in the presentation with real-world examples; you can't test all the things, sometimes you can't even test the thing because of non-technical limitations such as unavailability of systems, budget, or failure of management to understand the impact on organizational risk -
- I was able to give shout-outs to great work I've run across in my journeys, such as Resilient Coders of Boston and technical projects like Mockiato and OpenTelemetry -
- Quite a few people hung out afterward to express appreciation and interest in the sushi menu of ideas in the presentation. They are why I work so hard on my research areas. -
- I have to stop saying "you guys". It slipped out twice and I was internally embarrassed that this is still a latent habit. At least one-third of the attendees were women in technology and as important as being an accomplice to improving underrepresented communities (including non-binary individuals), my words need work. -
A Few Community Feedbacks, Anonymized
- - -
-
-
-
- What must we test?
-
-
-
- Regressions, integrations, negative testing -
- Deliver what you promised -
- Requirements & customer use cases -
- Underlying dependency changes -
- Access to our systems -
- Monitoring mechanisms -
- Pipelines -
- Things that lots of devs use (security libraries) -
- Things with lots of dependencies -
- - What can't we test?
-
-
-
- Processes that never finish (non-deterministic, infinite streams) -
- Brute-force enterprise cracking -
- Production systems -
- Production data (privacy concerns) -
- "All" versions of something, some equipment, types of data -
- Exhaustive testing -
- Randomness -
- High-fidelity combinations where dimensions exponentially multiply cases -
- Full system tests (takes too long for CI/CD) -
-
A few thoughts from folks in Slack (scrubbed for privacy)...
- - -Anonymized community member:
- - -Writing up my personal answers to @paulsbruce’s hivemind questions yesterday evening: What can/should you test?
- - --
-
- well specified properties of your system, of the form
if A then B. Only test those when your gut tells you they are complex enough to warrant a test, or as a preliminary step to fixing a bug, and making sure it won't get hit again (see my answer to the next question).
- - your monitoring and alerting pipeline. You can never test up front for everything, things will break. The least you can do is test for end failure, and polish your observability to make debugging/fixing easier. -
What can't/shouldn't you test?
- - --
-
- my answer here is a bit controversial, and a bit tongue in cheek (I'm the person writing more than 80% of the tests at my current job). You should test the least amount possible. In software, writing tests is very expensive. Tests add code, sometimes very complex code that is hard to read and hard to test in itself. This means it will quickly rot, or worse, it will prevent/keep people from modifying the software architecture or make bold moves because tests will break/become obsolete. For example, assume you tested every single detail of your current DB schema and DB behaviour. If changing the DB schema or moving to a new storage backend is "the right move" from a product standpoint, all your tests become obsolete. -
- tests will often add a lot of complexity to your codebase, only for the purpose of testing. You will have to add mocking at every level. You will have to set up CICD jobs. The cost of this depends on what kind of software you write, the problem is well solved for webby/microservicy/cloudy things, much less so for custom software / desktop software / web frontends / software with complex concurrency. For example, in my current job (highly concurrent embedded firmware, everything is mocked: every state machine, every hardware component, every ocmmunication bus is mocked so that individual state machines can be tested against. This means that if you add a new hardware sensor, you end up writing over 200 lines of boilerplate just to satisfy the mocking requirements. THis can be alleviated with scaffolding tools, some clever programming language features, but there is no denying the added complexity) -
To add to this, I think this is especially a problem for junior developers / developers who don't have enough experience with large scale codebases. They are either starry-eyed about TDD and "best practices" and "functional programming will save the world", and so don't exercise the right judgment on where to test and where not to test. So you end up with huge test suites that basically test that calling database.get_customer('john smith') == customer('john smith') which is pretty useless. much more useful would be logging that result.name != requested_name in the function get_customer
the first is going to be run in a mocked environment either on the dev machine, on the builder, or in a staging environment, and might not catch a race condition between writers and readers that happens under load every blue moon. the logging will, and you can alert on it. furthermore, if the bug is caught as a user bug "i tried to update the customer's name, but i got the wrong result", a developer can get the trace, and immediately figure out which function failed
- - -Then someone else chimed in:
- - -- - -It sounds like you’re pitting your anecdotal experience against the entire history of the industry and all the data showing that bugs are cheaper and faster to fix when found “to the left” i.e. before production. The idea that a developer can get a trace and immediately figure out which function failed is a starry-eyed fantasy when it comes to most software and systems in production in the world today.
-
The original contributor then continues with:
- - -- - -yeah, this is personal experience, and we don't just yeet stuff into production. as far data-driven software engineering, I find mostly scientific studies to be of dubious value, meaning we're all back to personal experience. as for trace driven debugging, it's working quite well at my workplace, I can go much more into details about how these things work (I had a webinar with qt up online but I think they took it down)
-as said, it's a bit tongue in cheek, but if there's no strong incentive to test something, I would say, don't. the one thing i do is keep tabs on which bugs we did fix later on, which parts of the sourcecode were affected, who fixed them, and draw conclusions from that
-
Sailboat Retrospective
- - -Using the concept of a sailboat retrospective, a few things that I'd like to improve are below, namely:
- - -Things that propel us:
- - --
-
- Many people said they really liked the collaborative nature of hive minding and would love to do this again because it got people to share learnings and ideas -
- Reading the crowd in real-time, I could see that people were connecting with the ideas and message; there were no "bad actors" or trolls in the crowd -
- Space, food, invites and social media logistics were handled well (not on me) -
Things that slowed us:
- - --
-
- My presentation was 50+ mins, way too long for a meetup IMO.
-
To improve this, I need to: -
-
-
- Break my content and narratives up to smaller chunks, ones that I can actually stick to a 20min timeframe on. If people want to hear more, I can chain on topics. -
- Recruit a timekeeper from the audience, someone who provides accountability -
- Don't get into minutia and examples that bulk out my message, unless asked -
- - Audio/video recording and last-minute mic difficulties kind of throws speakers off
-
To fix this? Maybe bring my own recording and A/V gear next time.
- - Having to verbally interrupt people at the agree upon time-breaks in 1-2-4-All seems counter to collaborative spirit.
-
To improve this, possibly use a Pavlovian sound on my phone (ding, chime, etc.)
-
Things to watch out for:
- - --
-
- I used the all-to-common gender-binary phrase "you guys" twice. Imagine rooms where it would somehow be fine to say that, but saying "hey ladies" to a mixed crowd would be considered pejorative to many cisgender men. Everything can be improved and this is certainly one thing I plan to be very conscious of. -
- Though it's important to have people write things down themselves, not everyone's handwriting can be read back by others after, and certainly not without high-fidelity photos of the post-its afterward.
-
To improve this, maybe stand with the final group representatives and if needed re-write the key concepts they verbalize to the "all" group on the whiteboard next to their post-it.
-
More reading:
- - --
-
- Afterthoughts on Hive Minding -
- Crossing Cross-Functional Chasms -
- Liberating Structures: 1-2-4-All Method -
Probably not unlike you, every day I work with folks caught in a clash between organizational processes and technology imperatives. "We have to get this new software up and running, but the #DevOps group won't give me the time of day."
- - -Large organizations don't have the luxury of 'move fast, break stuff'; if they did, their infrastructure, security, financial, and software release processes would be a chaotic mess...far more than usual. But how does one 'move fast' without breaking enterprise processes, particularly ones that they don't understand?
- - -Enterprise, Know Thyself
- - -The answer is simple: encourage engineers to always be curious to know more about their environment, constraints, and organizational culture. The more you know, the more nimble you'll be when planning and responding to unanticipated situations.
- - -Today I had a call with a health care company, working to get docker installed on a RHEL server provisioned by an infra team. What was missing was that the operator didn't know that the security team using Centrify to manage permissions on that box required tickets to be created to grant 'dzdo su' access for a very narrow window of time. Additionally, the usual 'person to connect with' was off on holiday break, so we were at the mercy of a semi-automated process for handling these tickets, and because they had already put in a similar request in the past 7 days, all new tickets would have to go through a manual verification process. This frustrated our friend.
- - -The frustration manifested in the form of the following statement:
- - -- - -Why can't they just let me have admin access to this non-production machine for more like 72 hours? Why only 2 meaasly hours at a time?
-- Engineer at an F100 health care organization
My empathy and encouragement to them was to "expect delays at first, don't expect everyone to know exactly how processes work until they've gone through them a few times, but don't accept things like this as discouragements to your primary objective."
- - -If everything were easy and no problems existed, kind words might be useless. When things are not working that way, knowing how to fix or overcome them goes a long way, just like a kind word at the right time. We crafted an email to the security team together explaining exactly what was needed AND WHY, as well as an indication of the authority and best/worst case timelines that we were operating under, and a sincere thank you.
- - -Enterprise "DevOps" Patterns that Feel Like Anti-Patterns
- - -In my current work, I experience a lot of different enterprise dynamics at many organizations around the world. The same themes, of course, come up often. A few dynamics I've seen in play when enterprises try to put new technology work in a pretty box (i.e. consolidate "DevOps engineers" into a centralized team) are:
- - --
-
- Enterprise DevOps/CloudOps/infra teams adopt the pattern of "planned work", just like development teams, using sprints and work tracking to provide manageable throughput and consistency of support to other organizational 'consumers'. This inherits other patterns like prioritization of work items, delivery dates, estimable progress, etc. -
- Low/no context requests into these teams get rejected because it's slow/impossible to prioritize and plan based on ambiguous work requirements -
- The amount of control and responsibility these teams have over security and infrastructure systems the organization is often considered "high risk", so they're subject to additional scrutiny come audit time -
That last point about auditing, particularly the psychological impacts on 'move fast' engineers, cannot be understated. When someone asks you to break protocol 'just this one time', it's you that's on the hook for explaining why you took action to do so, rarely the product owner or director who pressured the engineer to do it.
- - -Technical auditors that are worth anything more than spit will focus on processes instead of narrow activities because to comb through individual log entries is not scalable...but verifying that critical risk mitigative processes are in place and checking for examples of when the process is AND isn't being followed...that's far more doable in the few precious weeks that auditing firms are contracted to complete their work.
- - -The More You Know, The Faster You Can Go (Safely)
- - -An example of how understanding your enterprise organization's culture improves the speed of your work comes from an email today between two colleagues at F100+:
- - -- - -Can you confirm tentative dates when you are planning to conduct this test? Also will it take time to open firewall, post freeze incident tickets can be fast tracked?
-- Performance Engineering at Major Retailer
This is a simple example of proper planning. Notice that the first as is for concrete dates, an inference that others also need to have their shit together (in this particular case because they're conducting a 100k synthetic user test against some system, not a trivial thing in the slightest). The knowledge that firewall rules have to be requested ahead of time, and to notify incident response that potential issues reported may be due to the simulation, not real production traffic, comes from having experienced these things before. Understanding takes time.
- - -Another software engineer friend of mine in the open-source space and I were discussing the Centrify thing today, and he asked: "why can't they just set up and configure this server with temporary admin rights off to the side, then route appropriate ports and stuff to it once it's working?" Many practitioners in the bowels of enterprises will recognize a few wild assumptions there, and in no way is this a slight of my friend, but rather an example of how different thinking is from two very different engineering cultures. More specifically, those who are used to being constrained as opposed to those who aren't often have a harder time collaborating with each other because they're reasoning is predicated on very different past experiences. I see this one a lot.
- - -DevOps Is an Approach to Engineering Culture, not a Team
- - -This is my perspective after only 5yrs of working out what "DevOps" means. I encourage everyone to find their own by having their own journey of curiosity, keyboard work, and many conversations.
- - -There is and never should be a DevOps 'manifesto'. As Andrew Clay Shafer (@littleidea) once said, DevOps is about 'optimizing for people', not process or policy or one type of team only. Instead of manifesto bullet points, there are some clear and common principles that have stayed the test of time since 2008:
- - --
-
- A flow of work, as one way as possible -
- Observability and Transparency -
- Effective communication and collaboration -
- A high degree of automation -
- Feedback and experimentation for learning and mastery -
Some of the principles above come from early work like The Phoenix Project, The Goal, and Continuous Delivery; others come from more formalized research such as ISO and IEEE working groups on DevOps that I've been a part of over the past 3 years.
- - -I don't tend to bring the "DevOps is not a team" bit up when talking with F100s primarily because:
- - --
-
- it's not terribly relevant to our immediate work and deliverables -
- enterprises that think in terms of cost centers always make up departments, because "we have to know who's budget to pay them from and who manages them" -
- Now that DevOps is in vogue with various IT leaders and just like the manifestation of Agile everywhere now, DevOps is perceived as 'yet another demand to do things differently from management', so after being restructured, engineers often have enough open wounds that I don't need to throw salt on -
- if this is how people grok DevOps in their organization, there's little I as an 'outside' actor can do to change it...except maybe a little side-conversation over beers here and there, which I try to do as much as appropriately possible with receptive folks -
However, as an approach to engineering culture, DevOps expects people to work together, to "row in the same direction", and to learn at every opportunity. As I stated at the beginning of this post, learning more about the people and processes around you, the constraints and interactions behind the behaviors we see, being curious, and having empathy...these things all still work in an enterprise context.
- - -As the Buddha taught, the Middle Path gives vision, gives knowledge, and leads to calm, to insight, to enlightenment. There is always a 'middle way', and IMO is often the easiest path between extremes to get to the place where you want to be.
- diff --git a/_posts/2020-05-24-in-search-of-behavioral-indications.html b/_posts/2020-05-24-in-search-of-behavioral-indications.html deleted file mode 100644 index e51e1d7..0000000 --- a/_posts/2020-05-24-in-search-of-behavioral-indications.html +++ /dev/null @@ -1,92 +0,0 @@ ---- -layout: post -title: In Search of Behavioral Indications -date: 2020-05-24 11:47:35.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- advocacy -- community -- teams -tags: [] -meta: - _thumbnail_id: '1339' -author: - login: managed-wp-migration-823cfc63 - - display_name: Paul Bruce - first_name: Paul Bruce - last_name: '' -permalink: "/blog/2020/05/in-search-of-behavioral-indications/" ---- - -Recent changes in an organizing group I'm involved in have given rise to questions in my mind about how well we're doing, not just in terms of outputs, but in terms of cultural characteristics.
- - -In the past, I've used the model that Westrum created for assessing organizational culture to help put puzzle pieces on the table, not only to "see a bigger picture" but also to see what gaps exist (i.e. "missing pieces"). Even when you do this at some point, time goes on, and the picture changes, new boundaries are defined, and new gaps form, so it's important to do this on a regular basis.
- - -
-
Personally, I have experienced more of the pathological characteristics, such as "messengers shot", "bridging discouraged", "...scapegoating", and "novelty crushed". Others probably have experienced a swatch of these and others.
- - -However, there is also a solid basis of improved and positive characteristics in this group, namely "modest cooperation" and "failure leads to inquiry". We regularly encourage verbally and async in chat, and have some of the groundwork laid for peer ownership of responsibilities and risks.
- - -What I like about the Westrum grid above is that each cell value is a pretty good label, categorically speaking, but it can be difficult to understand what to do or avoid concretely. Everything is new to someone at some point, and though I have some experiences with how to move and improve some of these aspects (e.g. from bridging tolerated to encouraged, or the levels of novelty management), there is power in group-think.
- - -For those and for other reasons, I'm in search of a list of "attractor" and "detractor" behaviors that the group can use to A) sample the aggregate feelings of the group, and B) use as concrete "hotspots" to either improve upon (if deemed negative) or maintain/safeguard (if deemed positive). So far I have:
- - -Detractors:
- - --
-
- Too much outspokenness, not enough chance to speak -
- No voice, no real representation -
- Exclusive grip on wheelhouses -
- No healthy peer ownership -
- Silos, low collaboration, low emotional bucket fill -
- Shutdowns, cutoffs -
- Rephrasings -
- "Splaining" and assumptive attitudes -
- Lack of recognition, lack of constructive feedback -
- Public admonishment, not encouragements or recognition -
- Assignments and effort that is toil or low impact -
- No/low mindful group facilitation -
- Unsafe/alienating behavior -
- Ego/hubristic reasoning -
Attractors:
- - --
-
- Encouraging good ideas -
- Hearing from everyone, not "it's obvious" -
- Always assign peer (and/or assists) -
- Prioritize "preventative maintenance" characteristics over double-time out -
- Purposeful attempt to understand; ask open questions -
- Rotate wheelhouses; opportunity to change -
- Pre-agreement on time spend / limits / timeboxes -
- Purposeful recognition / appreciations / thanks -
- Proactive communication of bandwidth; ownership of comms -
- When work is done/blocked, swarm help w/ permission -
- Obtain permission before asking for labor, time/physical or emotional -
I'm a big fan of introspection, starting with the person in the mirror, so I'm approaching this as a group ask to help me personally as a friend understand the group and myself more. However, I try not to ask people to do free work, certainly not work that doesn't benefit them in some way. As an exercise, it has it's own virtues not limited to me anyway, but I also think that the outcomes can lead us to a place of constructive discussion in the upcoming organizer's 'summit' (a 4hr quarterly Zoom) where we can all get a better picture and agree to how to improve the culture of the group.
- - -For a holiday weekend, I've already spent about 4hrs (not to mention tossing and turning) on this, so in many ways this has been a "long weekend" for me.
- diff --git a/_posts/2020-09-02-more-like-water-less-like-waterfall.html b/_posts/2020-09-02-more-like-water-less-like-waterfall.html deleted file mode 100644 index cdd9787..0000000 --- a/_posts/2020-09-02-more-like-water-less-like-waterfall.html +++ /dev/null @@ -1,37 +0,0 @@ ---- -layout: post -title: More Like Water, Less Like Waterfall -date: 2020-09-02 21:23:08.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- personal-log -tags: [] -meta: - _thumbnail_id: '1354' -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2020/09/more-like-water-less-like-waterfall/" ---- - -It's been too long a time since I published something here. The more time I commit to professional and volunteer and personal projects, the less time time I feel I have to write. What a bullshit excuse too, because I book time on my calendar for other operational things, why not this? All it takes is diligence and sticking to a scheduled time.
- - -In tech, people use the word waterfall like a curse word due to historical reasons, a derogatory label...but an actual waterfall is a continuous stream that doesn't repeat itself. I want to be more like water, as Bruce Lee said, but for more than the reasons he had.
- - -As Master Lee indicated, we should be ready to change, like water in a glass conforms to the situation around it, in martial arts being rigid and stiff leads to being slow and overly anticipatory. Pangai-noon, translated as "half hard, half soft" also indicates that we need to keep strength and application in balance with speed and adaptability. Mindfulness also plays a huge part in this, living in "the now", being present in each moment, like water that completely fills every dip and cleft from the riverbed to the edge, but also constantly seeking balance at the surface.
- - -Anyway, I hypothesize that it will increase my mindfulness to exhale my field thoughts and experiences (minus personally identifiable stuff of course) to this blog on a consistent basis. I will dedicatedly try this for about 3 months, writing at least twice a week, and not give up because I missed one or two of these personal appointments. I will simply regale out what learnings I can from the main areas of my daily work, since it is so broad. If these pseudo-minutes strike on a topic meaty enough to write as it's own post, fine. If not, fine.
- - -It will, at the very least, force me to share something, at least, on a frequent basis. Less like 'waterfall' deployment as they say in software, more [continuous] like water.
- diff --git a/_posts/2020-09-02-personal-log-2020-09-02.html b/_posts/2020-09-02-personal-log-2020-09-02.html deleted file mode 100644 index 4074f76..0000000 --- a/_posts/2020-09-02-personal-log-2020-09-02.html +++ /dev/null @@ -1,90 +0,0 @@ ---- -layout: post -title: Personal Log 2020-09-02 -date: 2020-09-02 22:54:44.000000000 -04:00 -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: -- personal-log -tags: [] -meta: {} -author: - login: pbruce - - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: "/blog/2020/09/personal-log-2020-09-02/" ---- - -Started later than I'd like, trying to spend more good morning time with the kiddos. Nothing huge on the schedule, some customer and internal meetings, some time for deep work.
- - -Morning Popcorn
- - -Started to spin up revisionary work on an integration between qTest and NeoLoad Web. Good lord I need to document my pre-documentation work better for myself. Critical commands and obscure UI workflows are killer in products you don't know or care to know.
- - -Had to remind someone how to get to competitive intel docs. LMGTFY also applies to internal docs for organizations that use G-Suite.
- - -Redirected someone from using the old product requests system (Trello) to new one. Only saw this because I 'watch' everything in Trello and it comes via email summaries. Trello and email are gross, but hey, it worked today.
- - -Responded to a net-new person interested in collaborating on the book. Good sign that this channel isn't just a point-in-time blast, but an ongoing consideration from the channel admin now.
- - -Combined two email threads about the same topic from two different people groups in our org. Always nice when you can help people realize they're asking for or working on the same things as each other.
- - -Helping to Prioritize Tactical Feature Requests
- - -Hopped on with head of PM to discuss how to better prioritize tactical requests in the new product feedback and request system. Using a similar methodology as RICE (Reach-Impact-Confidence-Effort), my main suggestion was around how to represent urgency from the pre-sales or CSM side related to the topic, which isn't really covered explicitly enough to successfully operationalize a prioritization model that works for the whole business. Well received and there other practical things such as customer, Salesforce links, revenue size and risk info that we would also have to provide as metadata regardless of how we sum the urgency factor up into the high level view.
- - -Lunchtime Community Stuff
- - -In BDO Slack, got DMed politely about helping a local startup get some product feedback from our community. Points for asking what the best way was first, points for having the CEO (techy) do it, saw some community members positively engaging already. Also suggested that if they really want community love and help, sponsoring the upcoming DevOpsDays Boston event would be good, and sent along the prospectus. That's the nice thing about being an organizer in multiple groups, constructive forces.
- - -While I was on the topic and pivoted from lunch, a few other event organizer-y emails, sponsor asks, and an internal huddle about something that can't be ignored anymore.
- - -Deep Work, Fast CLI Fix, Short-Circuiting Flys
- - -Back to work-work: deep dive into the qTest integration. Looks like I have to completely create a Dockerfile from scratch (hello versioning hell) that includes their agent, not just the NeoLoad CLI and Python dependencies. Since their Docker example (stale, btw) was Ubuntu 16.04 based, struggled with getting Python 3.8/3.6 as default and proper pip requirements for almost an hour. Gave up and based off of Ubuntu 18.04 to simplify Python install process, and everything works much easier now. Also, their agentctl doesn't seem to have subcommands properly documented (wait, here it is, halfway down a sea of blah blahs), so there's no way to know if there's an automated approach to configuring an agent on a host (will try again tomorrow). So far, what I've learned is:
- - --
-
- Documentation is shit unless it's optimized for Google via keywords -
- No matter how much vendor documentation sites push their search bar on you, it's never as good as Google, and usually just downright awful -
- Cyclical documentation articles that bounce you back to high-level categories are bullshit -
- Documentation that is long enough to have sections and don't provide permalinks to those sections is candy ass -
- Documentation should be written for all supported platforms, not just old versions of Winblows -
Saw a product idea about the CLI come in from one of my best customers, made sense, so I implemented it, created a PR and pushed a pre-release version to address the issue. Customer and support comms, then feedbacks that it's better. Turn around time: less than 30mins.
- - -Figured out why I was getting a roles and permissions error deep in the qTest agent execution logs. Turns out, my trial to their platform had expired, so, nothing that seemed at all related between the error messages and the actual problem. Nice. Emailed our contacts to ask for proper technical partner license acquisition, hoping to hear from them soon. Reported the blocker to stakeholders.
- - -Short-circuited for the third time an ask from the CEO a fledgling startup who's been stalking Neotys and me about combining their tech with ours. They have one, maybe two customers, and the technology paradigms between them and us by definition are just so far from a match. They're just looking for legitimacy and customer lists. A big waste of our time, and I won't have it, I certainly won't waste my bosses time on it, though they have discussed and been dismissed in the past. Some people just don't understand when there's not something there, neither from a tech/architecture perspective nor a business/user case. We already have our priorities, this is a fly buzzing around the ointment at this point.
- - -Dinner and Event Organizer Meetings
- - -Back home for dinner and family time. Partner is listening to upcoming virtual classroom stuff for the new school year. Glad we and 20% of our community families, that's one whole elementary school out of the five in our city, emphatically said that they wouldn't be sending their kids back. At the beginning of the pandemic, we flattened the curve about demand on hospitals, why the hell would we all be expected to forget that the same applies with our kids and throwing them back into the schools all at the same time, even if that's for half-days (which btw doesn't help working parents more than stay-at-home by much).
- - -Back to DevOps community organizing, event sponsors working group 30mins weekly until the event is done. There are very few 'other fruits to squeeze' for sponsorship dollars this year. I guess that makes it all the more meaningful, the companies and organizations that have committed already. The fund for next year will be okay, we won't be adding to it this with this year's revenue for sure, but at least we reached a milestone precedent by committing to donate all the net ticket revenue that comes into good causes! Boy, I worked for months getting alignment between groups on how to make that happen, and along with another organizer to put the right pressure words on the right people at the right time, we now we have a list of causes and agreement across groups that it will happen. We also have a process and clear criteria that can scale and be reused again next year. We will also have data about how many people 'feel generous' when offered an option to give more than the recommended ticket price, and data on how many people pick a free ticket when faced with the option that 100% of their contribution will go to something altruistic. We will need to write a retro, draft it up beforehand to publish the day after, about how much money and who it went to; this is not something that can wait for weeks or months like the AV post-production in prior years. People will want to know.
- diff --git a/_posts/2021-12-06-5-trends-shaping-the-future-of-performance-engineering.markdown b/_posts/2021-12-06-5-trends-shaping-the-future-of-performance-engineering.markdown deleted file mode 100644 index c14dc89..0000000 --- a/_posts/2021-12-06-5-trends-shaping-the-future-of-performance-engineering.markdown +++ /dev/null @@ -1,316 +0,0 @@ ---- -layout: post -title: 5 Trends Shaping the Future of Performance Engineering -date: 2021-12-05T15:47:35.000Z -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: - - performance - - testing - - engineering - - continuous - - DevOps -tags: [] -author: - login: pbruce - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: /blog/2021/12/5-trends-shaping-the-future-of-performance-engineering/ ---- -As part of my usual duties, Tricentis asked me to name a few key themes and trends - that I've seen during customer engineering work with forward-thinking customers, - ones that will likely be a big part of performance engineering in 2022. - -There is no predicting the future, there's only keeping your eyes and ears open - and developing intuition, then collecting evidence for AND against hypotheses. - -You can signup/view the recording of this webinar [here](https://www.bigmarker.com/techwell-corporation/5-Trends-Shaping-the-Future-of-Performance-Engineering) - - - -- [1. Resiliency and controlled chaos engineering by default](#1-resiliency-and-controlled-chaos-engineering-by-default) -- [2. Deep and wide telemetry for all cloud resources](#2-deep-and-wide-telemetry-for-all-cloud-resources) -- [3. Industry standardization of metrics and instrumentation](#3-industry-standardization-of-metrics-and-instrumentation) -- [4. Intelligent readiness pre and post deployment](#4-intelligent-readiness-pre-and-post-deployment) -- [5. A common approach across custom and packaged apps](#5-a-common-approach-across-custom-and-packaged-apps) - -# 1. Resiliency and controlled chaos engineering by default -Chaos engineering has been around for a while, but it often takes quite a while for - good ideas and practices to mature in the crucible of large at-scale organizations. - -Many of my largest customers have some form of failure more injection in play during - large load tests, often using tools like Gremlin or the Chaos tools from Netflix - to simulate infrastructure flakiness and unavailability. This is mostly in lower - environments, though some have small pockets of already-rugged services in production - that are regularly rotated and decommissioned as part of a plan. - -They also utilize load testing because there's very little you can defensibly glean - from injecting chaos/faults but then manually testing things out as only one user. - Load testing, even at non-massive volumes, provides statistical relevance to errors - observed and potentially correlated to chaos exercises. - -After participating in a small back-and-forth on DevOps Unbound episode with the very - awesome [Tammy Bryant Butow](https://www.linkedin.com/in/tammybutow/), it's clear that - performance and reliability teams must - work together to advocate and grow both performance testing and chaos engineering - practices together; there's little point in doing either without seriously considering - the other. Yes this means I'm saying that load testing alone is not enough; it's - a good first step to exercising your systems pre-release, but the next step should - use chaos/chaotic fault injection to really test your system's true failure states. - -A few resources if you'd like to hear more about chaos/resiliency and performance testing: -- [bit.ly/chaosdeck : A slide deck I use for tech intros and workshops](https://bit.ly/chaosdeck) -- [bit.ly/chaosvideos : A playlist of really good videos on chaos testing](https://bit.ly/chaosvideos) -- NeoLoad + Gremlin: [Webinar](https://youtu.be/K-MxfuzOdU8) and [White Paper](https://d28h099uturm62.cloudfront.net/wp-content/uploads/2021/03/Datasheet-NeoLoad-Gremlin-EN.pdf) - -# 2. Deep and wide telemetry for all cloud resources -In the past, we engineers have always needed more information to diagnose issues. - This often took classic forms of monitoring servers directly: [CPU, memory, disk, and network](https://youtu.be/YlAqs8cWP2I?t=272) utilization. Though sometimes this kind of direct access is still necessary, it's problematic from - a security and cloud-based infrastructure perspective to allow this. It's far more - common now to see application performance monitoring (APM) agents take the responsibility - of being on-server, in-process, actors that safely ship both classic and deeper telemetry about - the state of our servers ***AND*** service health back to a central system (e.g. Dynatrace, Prometheus, DataDog, etc.) - -But going deep isn't always enough. In distributed systems and deployments using - shared infrastructures (think blade servers, cloud racks, Kubernetes clusters), - it's also just as critical as 'going deep' to also be able to 'go wide'. In other words, - measuring as many of these shared components as feasible, specifically because - the parts affect each other. One pods spike is another pods noisy neighbor. No - cloud VM's spec is ***guaranteed***, it's all best-effort at the end of the day. - -There are also 'wide metrics', such as number of pod or VM restarts, that only a - management layer captures since the individual component in this case dies unexpectedly. - I also include networking infrastructure such as security firewalls, software-defined - routers, and load balancers into 'wide metrics' because they often cause issues with - performance yet serve many services. Not having access to firewall telemetry that - correlates to socket timeouts in a load test can be maddening because you can't - prove or disprove if that particular device is part of the issue or not. - -Even (and sometimes especially) with SaaS providers whom you trust to fulfill their SLAs, - their Ops teams have implemented rules for detecting and rate-limiting, which don't - show up until you start hitting them. From [my work](https://paulsbruce.io/blog/2019/08/on-lack-of-transparency-in-saas-providers/) with clients using various Salesforce - SaaS platforms, 'governers' (their name for these rules) are usually undocumented publicly, - and sometimes not documented well or even at all internally, and diagnosing why - you're hitting performance limits on their SaaS platform(s) takes weeks of back-and-forth - with their technical teams. - -The point is, the more telemetry we have, both deep into the components and wide - across runtime management layers, the more likely we can diagnose what the biggest - contributing factors are to a performance or reliability issue. - -The future of telemetry is looking better and better year after year though. Every - cloud provider typically has (low cardinality, low granularity) metrics about resources - you use there. More organizations are expanding or shifting their APM platform investments - to support cloud-based, not just in-premise, resources. And as Kubernetes adoption - continues to grow exponentially, APMs and CNCF tooling (such as eBPF) is getting - deeper and wider about providing centralized reporting for telemetry. - -Some resources to learn more about deep and wide telemetry: -- [PromCon EU 2019: Containing Your Cardinality](https://www.youtube.com/watch?v=49BGvC1coG4) -- [PromCon 2018: OpenMetrics](https://www.youtube.com/watch?v=Za7vmKr11cQ) -- [Metrics for Kubernetes System Components](https://kubernetes.io/docs/concepts/cluster-administration/system-metrics/) -- [Blog: On Lack of Transparency in SaaS Providers](https://paulsbruce.io/blog/2019/08/on-lack-of-transparency-in-saas-providers/) -- [Prometheus Query Examples](https://prometheus.io/docs/prometheus/latest/querying/examples/) - -This leads us nicely into the next topic... - -# 3. Industry standardization of metrics and instrumentation -Now that there are so many ways to collect telemetry, inject tracing, and correlate logs, - the choice for which technologies and platforms your organization should support - is increasingly complex. Each team has their own preferences for which tools and - technologies to implement, even if there are prescribed 'best practices' and already - paid for platforms. - -And if you were to propose that one of your important services change how it captures and - ships telemetry (often a massive rewrite for legacy codebases), you'd likely face a huge - battle with product teams who already have an overbooked backlog of both new features and - tech debt items to take up their time. So how can we make this better moving forward? - -The short answer is for the ***industry to standardize*** at least some of the core - structures, formats, and SDKs for instrumenting and embedding telemetry code into - your apps and services. A *huge* amount of effort has been going on for years - in the CNCF OpenTelemetry (OTel) project for this exact reason. Utilizing OTel - for all greenfield/new development is a great start, and it's not that terribly hard - to retrofit for legacy systems instrumentation as well. - -And 'observability' is not the only buzzword getting standardization love these days. - Remember when APIs were the new cool kid on the block? Well beyond the API definition - wars from 2015, we now see people realizing the importance of standards around - measuring non-functional aspects of APIs as well, with the recent formulation of - ['The API Rating Agency'](https://thenewstack.io/api-rating-agency-brings-consistency-to-api-measurements/). - -There's even a high-level standard for enterprise DevOps principles and practices - (that I and others worked on) called IEEE 2675-2021. I specifically made sure that - the vision and outcomes statements of the Measurement, Risk Management, QM, QA, verification, - and validation process sections had ***HEAVY*** helpings of how to accelerate telemetry - and synthesis back in to decision processes. - -But staying with performance and reliability, OTel is definately worth getting up to speed on. - To learn more about the CNCF OpenTelemetry project, here are a few resources: - -- [o11yfest.org](https://o11yfest.org) (free signup for two days of presentations) -- [Observability Primer, LYIT Guest Lecture, Paul Bruce](https://www.youtube.com/watch?v=C1nvcLapcyA&list=PLFXQmSmq7uXQU3IrbypXKetf5Tos_XWOU) -- [The New Stack: API Ratings agency...](https://thenewstack.io/api-rating-agency-brings-consistency-to-api-measurements/) -- [Announcing IEEE 2675-2021: DevOps Standard to Build Secure and Reliable Systems](https://www.youtube.com/watch?v=8Og8rhjqgQo&list=PLFXQmSmq7uXQU3IrbypXKetf5Tos_XWOU&t=63s) - -# 4. Intelligent readiness pre and post deployment -When we talk about being 'ready', it matters what we precisely mean. What does it - mean for you to be ready to release to production users? How is this different - than being ready to deploy to a production-like environment for load testing? - Who defines what 'ready' means in these and other contexts? - -What we're really talking about in most of these cases is ***confidence*** people - have that what they're about to do will have the anticipated affect on - users or the business under an appropriate level of risk (never zero). - -The simplest answer I can provide is that 'ready' should be: - -- ***clearly defined***, in terms of - - process to complete correctly - - evidences of satisfactory success - - risk mitigation plan, should issues arise -- ***measurable*** in concrete terms, such as - - SLAs, SLOs, SLIs agreed to by all stakeholders - - both before and after release to production users -- ***communicated*** to all stakeholders - - what the rollout plan is - - what success signals to look for - - what failure signals to look for - - how to raise your hand if something doesn't look right -- ***actively managed*** by a product team lead - - but also include on-call plan for devs and infra/ops - - and include appropriate status updates to stakeholders - - and who never assumes that things are working as planned - -Okay, so for performance engineering, what does 'intelligent' readiness look like? - Well there are a lot of the points above that sound like they have to be done - by humans (who are intelligent), but many of them are patternable and repeatable - once some of the basic thinking is laid down, and so can and should be automated. - -Performance tests shouldn't all have to be special-snowflake, big-bang, complex - operations. In fact, most performance testing should be frequent and small, focusing - on API, sanity, and low-volume validations early on in the development lifecycle. - -As such, it is easy to 'clearly define' what the performance testing process is, - both on your organization's wiki or knowledge management platform, and most certainly - in the automated pipeline scripts that inscribe the repeatable process into your - continuous delivery motions. - -All testing, performance or otherwise, should produce clear and trustworthy ***exit signals***, - both in the case of successful completion and unsuccessful. Usually my clients - use load test SLAs/thresholds around error rate and response times under sustained - load to determine if the test should keep going or 'fast fail'. If the load test - infrastructure or project sources have issues, this would be what I call a 'catastrophically unsuccessful' test, as opposed to a test that completed but didn't meet the final - performance acceptance criteria that the team agreed to. Except for short, passing sanity - tests, all test runs and results should be retained for an appropriate amount of time. - -Getting 'measurable' readiness means that you can demonstrate evidence not only - before deployment and release to production, but afterwards as well. Some people and - systems lend themselves to testing in production more than others, and most clients - I see 'load testing in prod' are usually doing it against production resources that - have not been rotated in to use by production users. This kind of capability takes - rollout and deployment maturity many organizations still lack, so many just opt - for pre-release load testing in either production-like staging environments or - something close to that. The point is that the same testing you do in lower environments - should be easy to switch over to other higher environments that need a pressure - check before *releaseing* to production users. - -***Communicating*** these efforts is important because when a load test runs somewhere, - someone will definitely feel it, whether that's other test engineers using that environment - or the infra/ops team getting alerts. This is usually why you don't trigger big - performance tests on every single code check-in, not to mention the time budget, - the system under test usually takes resources, and if shared/permanent impacts others. - -***Active management*** of performance engineering tasks such as big load tests - isn't hard; this is usually the product team asking for someone to run the test. - What's harder is to build continuous performance testing practices that communicate - success and failures back to the product teams A) at the right times, and B) with - the right level of actionable detail. - -There is rarely a 'silver bullet' in performance - engineering, no single 'root cause'...until you step through things to find a specific - issue causing lots of symptoms. Sure, 90% of the time, it's the database; but then - there are times it's not. What people often mistake as 'single root cause' situations - are actually a biggest contributing factor that's loud enough and that you happened upon - first. There are always more than one contributing factor, and once we find something - we're often too busy to look around for the others. - -The point is here, there's no one optimal performance report format or easy way - to identify 'root cause' in all cases, not without expertise and experience guiding it. - There are however common and practical points of information in load tests, such as: - -- metrics perceived by the load platform - - RED signals: rates, errors, durations -- metrics observed via monitoring - - USE signals: server/service side Utilization, Saturation, and Errors - - Wide metrics: monitoring storage, management layers, and network devices -- notable moments and in-cycle events: - - chaos testing activities - - re-balancing or optimization rules being triggered - - real-time manual configuration changes - -By visualizing these together, calling out 'interesting moments', and constantly - taking action to automate known faults and anti-patterns into actionable outcomes, - product teams can effectively move performance testing into continuous delivery. - - -# 5. A common approach across custom and packaged apps - -I often hear that "the API teams are doing their own thing" or "testing the - apps/services we build is very different from 'commercial of the shelf' (COTS) products." - I know what my clients are talking about, I get it. Testing Workday and JIRA - is very different than that your organization's e-commerce site or claims processing app. - -Mostly it boils down to: - -- varying levels of 'testability' in the Systems Under Test (SUT) -- low expertise or experience with the intricacies of the SUT -- lack of visibility into the back-ends of the COTS apps (see prior comments about Salesforce) -- low/no direct interaction or communication with SUT dev team(s) - -Especially with packaged apps, it's not so much the app itself that's causing - performance issues, but rather all the customizations and the operational environment - configuration that fundamentally change the definition of what a COTS vendor does - to verify the performance and reliability of their core deliverables vs. how it - actually works for your organization as deployed. If you've ever gone through an - SAP enterprise implementation, you might be familiar with their 'configuration testing' - which includes load testing once customizations and environment are loaded up. - -With custom apps, you usually have more of an ability to reach out to teams internally, - even if they're contracting, to ask questions. There are often architectural diagrams - and systems (of record, such as APM platforms) already in use that you simply have - to ask for access to and explain why it's important to the work you've been asked to do. - If you find untestable situations, call them out, make sure you add a testing user - story to the product team's backlog and get the PO and Product Management to assess - the risk it represents. If they can't do that, call your security/compliance office. - -In both custom app and packaged app cases, you can always make progress on at least - two of the four bullet points above, immediately. It might not be easy...certainly getting - interactions with a JIRA dev team at Atlassian is far harder than say sending - a virtual 30min meeting invite to an internal Product Owner (PO) and their lead - developer. Sometimes the required level of expertise with a complex web app takes - longer than provided to build, and that's not an easy discussion to have with some - engineering managers. And of course it's easy to blame tools and other people. - -A bad engineer blames their tools. It is not the tools we use that make us good, - but rather how we employ them. - -How we employ testing tools should be guided by what approach to testing we have - decided to take. If something proves to be inordinately hard to test, that's - an indication to go investigate how others have solved the problem, or ask the - persons responsible for building it how they would approach validating that it - works well. - -In a nutshell, performance engineering has been and even more needs to consolidate - their expertise, their practices, and their tooling to scale it to others in their - organization (don't forget that continuous pipelines are like another person too). - -# Summing It All Up - -In my customers and clients, I see these trends growing, some better/faster than others. - It is encouraging to see this, but also compelling me to encourage others to grow - and improve their performance and reliability practices. diff --git a/_posts/2021-12-09-accounting-for-privilege.markdown b/_posts/2021-12-09-accounting-for-privilege.markdown deleted file mode 100644 index 4b8401b..0000000 --- a/_posts/2021-12-09-accounting-for-privilege.markdown +++ /dev/null @@ -1,135 +0,0 @@ ---- -layout: post -title: Accounting for Privilege -date: 2021-12-07T21:43:47.000Z -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: - - "DevOpsDays Boston 2021" - - "event coverage" - - "community" - - volunteering - - DevOps -tags: [] -author: - login: pbruce - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: /blog/2021/12/accounting-for-privilege/ ---- - -After a few years of volunteer organizing [DevOpsDays Boston](https://devopsdaysbos.org/) and other local tech events, I found that there - were some things I wanted to work out personally in other sandboxes, but still drive myself to "provide - more value than I consume". Specifically, I want to put my money where my mouth is regarding under-representation and white privilege in the tech industry. So I did, with a lot of help from others. - -This year I can report that ***$21,000 from [DevOpsDays Boston](https://devopsdaysbos.org/)*** and around ***$17,000 from [o11yfest](https://o11yfest.org)*** have been donated to a combination of [Resilient Coders](https://www.resilientcoders.org/), [KodeConnect](https://kodeconnect.org/), [YearUp](https://www.yearup.org/), [Black Girls CODE](https://www.blackgirlscode.com/), [Stop AAPI Hate](https://stopaapihate.org/), [COVID Relief Fund for India](https://aidindia.org/donate/covid-relief-fund/), and [Trans Lifeline](https://translifeline.org/). Some in combined effort, some with heavy personal effort, but every dollar here bears the organizing groups' willingness and effort to go above and beyond. - -- [Jump to the Final Distribution and Tallies](#the-final-distribution-and-tallies) - -# How Did We Arrive Here? - -If I were to retroactively reconstruct my general thinking, it has been: - -1. I didn't set out to help to drive white privilege, but once I realized how bad it is, I can't ignore my responsibility to do something meaningful about it -2. I want to bring other voices and perspectives to the table, so the less I talk/speak/present and the more I can help under-represented folks do that, the better off we all are -3. Moving into primarily organizing role(s), another layer of privilege, this helps with the above but should also inherit the same motion to ["step up by stepping aside"](/blog/2016/02/7-practical-tips-for-inclusion/) in the long-term -4. The ideas and hypotheses about how to do this stuff must be informed by those who are under-represented, not just my empathy or inclusion assumptions -5. Some motions and approaches need trial-and-learning cycles before expecting them to work in the broader context of the [DevOpsDays Boston](https://devopsdaysbos.org/) event organizer's group - -Ultimately, after countless organizing and board meetings, policy discussions, delayed emails, and other legitimate but frustratingly complex dialogs, I realized that I could run outside experiments myself, find what works and what doesn't there, and then synthesize that back in to the bigger groups I work with. - -# Why Money, Isn't That Just Another Tech Bro Handout? - -I don't think of myself as a 'tech bro'. I don't: -- like to hang out with people who just look, sound, think, or act like me or agree with me -- have every day of the week available to just stay 'in the city' for a late meetup -- constantly try to advance my career by stepping on others -- go around trying to fix people simply because I can code -- talk or shout people down, like, ever -- assume that I will by default own things (or matter) in the future - -I do: -- err on the side of self-less -- make mistakes and look to correct them when identified -- care about the well-being of others, usually more than my own :( -- everything I can think of to encourage non-white-non-bros to elevate and be treated equitably, both in my full-time job and in my volunteer groups -- read, listen, and absorb as much as I can possibly take about: - - historical inequities - - unconscious bias - - gender equality - - non-binary and gender-neutral identity -- bear a HUGE debt of guilt for the racism that is the entire history of the United States of America - -In short, I'm learning and taking responsibility for what I can and should do to enable and accelerate others to help change the equations. It's more than I can say for some, but still never enough for the size of what I feel. - -In 2019 (remember the days of old), some of the 'best thinking' in the group was to offer free 'under-represented' tickets. This never sat well with me, though as one of the group willing to try things, I did my part to visit various meetups and asked the organizers if/how it would be possible to encourage people to take them. We had tracking by code so we know that even after many of the 50 tickets were given out locally, this didn't really compel people to come to some local tech event. And not to mention, this was putting organizers in a place of power over 'who to give them to', how 'under-represented' do you have to be to deserve one, etc. Blech. That model was firmly in 'handout' territory and disappeared quickly. - -Then COVID crashed the world and 2020's event went virtual, which actually freed us up to offer tickets at a sliding scale, including a free option. The fear from others was that we would get people who would have otherwise paid getting free tickets instead. This was exacerbated by using Hopin, a ticketing + virtual event platform, which at that time (and since last I checked in May 2021) had no way to set defaults and sort order on how different ticket types appeared for people. Either way, the tech world was only starting to realize at that time that all events would be virtual indefinitely. So though it wasn't perfect, 2020 ticketing aired on the side of inclusive (defaulting to 'free'), and no one had to get in the middle of 'who deserves a free ticket', which itself was a huge improvement. - -So fast forward from 'free tickets' to directly generating $17,000 for important causes, no, this money isn't a handout. I consider it the first repeatable model for what reparations and preparations are due other folks and communities, more than I can personally afford any other way. And in some small way, it's my own way of dealing with guilt and the need for rightly-human recompense about things like this: - -[]({{ site.baseurl }}/assets/images/2021/12/20211208_121234.jpg) - -***[From "They Called Us Enemy" p.24-25 - George Takei, Justin Eisinger, Steven Scott, Harmony Becker](https://www.amazon.com/They-Called-Enemy-George-Takei/dp/1603094504)*** - -Also, two of the three good causes have already reached back out to us to personally thank us AND to figure out how we can collaborate together in 2022! This is already part of my charter as part of the [Boston DevOps Network](https://bostondevopsnetwork.org) board who underwrites the [DevOpsDays Boston](https://devopsdaysbos.org/) event, and I'm excited to develop this interest into action and tangible positive impact moving forward. - -# The Breakdown: How Did This Work? - -Conferences, virtual or otherwise, require capital to start and complete. Vendors in a virtual conference are online platforms (like Restream, Ti.to Ticketing, Vito.Community, Live Captioning persons, Otter.ai for alternative Transcripts, Zoom Pro for breakouts, etc.) and logistics (like A/V, Graphic Recording, creatives artistry, swag/distribution, MoneyOps/AR/AP). This stuff costs real money. I was able to do it with about $15k all told. At times I had to shift personal money into my LLC account as CapEx to cover one or two things before receiving sponsor money. I only ever wanted to break even, and mostly focus on how to turn interest (attendee tickets and sponsorship overflow) directly into donations to good causes. - -So that's why (at least for virtual [o11yfest](https://o11yfest.org)), I only needed three 'premiere sponsors' (total $21k). Then I can encourage attendees as much as possible from their hearts to donate, with a default of $30, but also provide a free ticket option to include those that really couldn't afford it otherwise. The rest of the tech companies who inquired about sponsorship after the premiere spots were gone, I was able to encourage to 'go directly donate a minimum of $2k to one of these good causes, just forward me the email proof of donation, and you get gold level perks'. Five startups took me up on this, an easy way to attach their names and logos to a community-driven event, generating a total of $10k in donations (which I didn't have to accounts-receivable for, less to do and get fees taken from). Finally, more than 2/3rds of the attendees opted to donate something, many the default of $30 or more per ticket, totaling about ~$5,400 in net ticket sales (minus the platform percentage) which was all by definition earmarked for donations. - -# What Worked Well - -A lot, surprisingly, and many because of the group perspective, group effort, and virtuous individuals. - -- heavy emphasis on inclusivity, from the organizing groups to the expectations to premiere sponsors, the graphics and the content/presentations; I'm grateful and proud of what we accomplished here -- having an 'entourage' of organizers; even if I did a lot of the General Manager stuff, it always comes down to a team effort at the actual end of the day -- paying for one of my favorite independent musical artist to do a special COVID-remote show for us, aside from supporting them financially, it put FUN on the menu and lots of great feedback about it afterward -- commissioning a young local graphical artist to come up with 'mascot' art; lots of little and different robots that could be used in everything from digital platform themes to cards to hoodies and gift boxes to fridge magnet sets. they were everywhere and clearly identified how unique [o11yfest](https://o11yfest.org) would be and was. - - -# What Didn't Work As Well - -Oh there were so many things, but here are the ones worth sharing: - -- Don't bother with virtual swag bags/boxes mailed to people, the costs and logistics are are not work the 'cutsie' effect (I only did this because it helped people not feel so disconnected during COVID) -- Hoodies...rather than commissioning, storing, distributing centrally through a vendor, there are plenty of online self-service options where attendees who want to buy their own can pay and keep their address and PII to themselves. The only hoodies I did were 30, for speakers and volunteers, and that was exhausting to deal with, but then I got to hand-write unique thank you notes to each of them :) -- Paying lots of money for dedicated server for 'the afterparty' platform (gather.town); I needed to make sure people weren't denied access in a free account, but like 30 people showed up and I budgeted for 300. - -# The Final Distribution and Tallies - -| From | To | How Much | Why | -| [DevOpsDays Boston](https://devopsdaysbos.org/) | [YearUp](https://www.yearup.org/) | $7,000 | organizers disbursal | -| [DevOpsDays Boston](https://devopsdaysbos.org/) | [KodeConnect](https://kodeconnect.org/) | $7,000 | organizers disbursal | -| [DevOpsDays Boston](https://devopsdaysbos.org/) | [Resilient Coders](https://www.resilientcoders.org/) | $7,000 | organizers disbursal | -| [DevSecOps Days Boston](https://devsecopsdaysboston.org) - synopsys | [Resilient Coders](https://www.resilientcoders.org/) | $2,000 | contributor sponsorship | -| [o11yfest](https://o11yfest.org) - Harness.io | [Stop AAPI Hate](https://stopaapihate.org/) | $2,000 | contributor sponsorship | -| [o11yfest](https://o11yfest.org) - Chronosphere | [Stop AAPI Hate](https://stopaapihate.org/) | $2,000 | contributor sponsorship | -| [o11yfest](https://o11yfest.org) - attendees | [Stop AAPI Hate](https://stopaapihate.org/) | $500 | balance of net ticket sales | -| [o11yfest](https://o11yfest.org) - attendees | [Trans Lifeline](https://translifeline.org/) | $4,500 | balance of net ticket sales | -| [o11yfest](https://o11yfest.org) - StackPulse | [Black Girls CODE](https://www.blackgirlscode.com/) | $2,000 | contributor sponsorship | -| [o11yfest](https://o11yfest.org) - FireHydrant | [Black Girls CODE](https://www.blackgirlscode.com/) | $2,000 | contributor sponsorship | -| [o11yfest](https://o11yfest.org) - SLOConf/Nobl9 | [Black Girls CODE](https://www.blackgirlscode.com/) | $2,000 | contributor sponsorship | -| [DevSecOps Days Boston](https://devsecopsdaysboston.org) | [COVID Relief Fund for India](https://aidindia.org/donate/covid-relief-fund/) | $672.35 | 100% of net ticket sales from ***[DevSecOps Days Boston](https://devsecopsdaysboston.org)*** | - -# Shout-outs and Contributors - -A special thanks to [Liz Fong-Jones](https://twitter.com/lizthegrey) who was my co-chair of [o11yfest](https://o11yfest.org), notwithstanding all the other things she had to do this summer. Gracious, patient, concise, and hella-connected. Thank you for your meaningful guidance and support! - -[Michael Thomas Clark](https://www.linkedin.com/in/michaeltclark/), another volunteer core organizer from [DevOpsDays Boston](https://devopsdaysbos.org/), was my financial email "plus ones" person, and a regular at our organizing weekly check-ins. Kind of my conspirator on the donations thing, we are of the same spirit that serious donation strategy should be part of the ethics of every tech event, at least the ones we're volunteering with. - -[Kate Ruh](https://www.linkedin.com/in/kate-ruh/), who helped with DevSecOps Days on a whim, and then totally came through and is now one of the [DevOpsDays Boston](https://devopsdaysbos.org/) organizers! So insightful, effective, efficient, and generative. - -[Amelia Mango](https://twitter.com/ameliamango), for the second year in a row, prompted me to get off my ass and do something! Not only a proven good actor in the OTel and API spaces around intelligent marketing, she also helped a lot with ideation, logistics and just keeping the ball rolling on tactical work as bigger general management things started to take all my time up. Always interested in working with Amelia about, well, anything. - -[Ruth Lennon](https://www.linkedin.com/in/ruth-g-lennon/), a wise and fearless leader in the standards and higher ed technology space. Ruth was co-chair of [DevSecOps Days Boston](https://devsecopsdaysboston.org), which generated ~$2,672 in donations as a first-year virtual conference. Her insight and guidance on topics and presentations for this event was instrumental to putting on a top-quality event. - -# It Really Takes a Village - -Again, none of this would have happened without a lot of peoples' concerted efforts, off-hours and completely volunteer. If you're interested in helping with these events or even just becoming part of the Boston DevOps or o11yfest communities, please reach out to me via [LinkedIn](https://www.linkedin.com/in/paulsbruce/) and let's chat! diff --git a/_posts/2021-12-29-defining-developer.markdown b/_posts/2021-12-29-defining-developer.markdown deleted file mode 100644 index fd097de..0000000 --- a/_posts/2021-12-29-defining-developer.markdown +++ /dev/null @@ -1,92 +0,0 @@ ---- -layout: post -title: Defining 'Developer' -date: 2021-12-29T11:43:21.000Z -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: - - developer - - teams - - engineering -tags: [] -author: - login: pbruce - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: /blog/2021/12/defining-developer/ ---- - -It may sound like an unnecessary errand in 2021 to have to define what "developer" (["programmer"](https://en.wikipedia.org/wiki/Programmer)) means, but once again I find myself in the complimentary and contradictory position of having to disambiguate what people in software mean when they casually say "dev" or "developer" in rooms of like-minded individuals. - -# TL;DR - -A 'developer' is someone who contributes functionality, typically in the form of code, to a product or service that runs in production to drive the material goal of an organization. This usually means: - -- Translating user stories into running software -- Further detailing requirements of above stories -- Estimating and/or preparing to implement changes -- Primarily adding new, then secondarily updating or fixing existing, functionality -- Writing unit tests which verify core code components of the above changes -- Consulting with other stakeholders (architects, users, operations engineers) -- Describing the material operating conditions of the functionality - -It does not primarily mean, though sometimes involves: - -- writing pipeline or infra-as code -- whiling hours away trying to get Kubernetes to do what you want it to do -- writing functional and ~~non-functional~~ operational tests -- clicking around through cloud consoles like AWS/Azure/GCP to get software to work - -In short, the 'magic' is consuming whatever it takes to produce software. I dare 'test-ers', 'DevOps', or product managers/owners to do this. I fucking dare them, and if they do, it will still be something a professional programmer has to rewrite so that it A) makes sense to anyone else, B) is maintainable once shipped, and C) makes everyone money. - - - -# A Developer's Best (Local) Goal - -The best goal of a developer is to safely and efficiently translate user needs into working software that achieves their organization's goal, preferably better than other attempts to do the same in other industryAnd examples. This is a ["local optima"](https://medium.com/prodopsio/devops-theory-of-constraints-cf1477f9bd1a#:~:text=A%20decision%20made%20based%20on%20local%20optima%20will%20in%20general%20not%20be%20as%20bad%20as%20a%20random%20one) or local goal and contributes to their team's less local one, which is the same as above, but with additional liberties and constraints that change the dynamics of how best to accomplish the goal. - -## A Developer's Dis/Optimal Approach - -Regarding the latter, though every individual in the team is subject to personal ambitions and inhibitions, it is [shitty politics](https://www.stellarperformance.com/articles/advice/stellar-leaders/politics-the-dirty-word.html) to live by them. In the past, I've experienced colleagues who do this, and not only do they contribute to [pathological](https://cloud.google.com/architecture/devops/devops-culture-westrum-organizational-culture) outcomes, they undermine their own effectiveness and sense of satisfaction from their work. - -Most developers, well, we work in teams. Even if we are "ICs" (individual contributors), we are surrounded by others that need what we produce and need us to be at our best, not just from an individual perspective, but for others around us. If you doubt this perspective, consider trying to get anything done without having working relationships with other team members that help you do the above and below bulleted activities. - -So, from a tactical perspective, yes, the above bullet points about what a developer does represents the 80% under the Bell curve of what IMO a developer does. As a life-long programmer (skill and curiosity-driven, not vocational 'developer'), and if you are in fact a developer, you may find yourself doing many other things like: - -- screwing around with an issue tracking systems -- contributing your thoughts to peer reviews -- playing video or board games with some colleagues -- learning about new technologies, usually on your own time/dime -- defending your position against product managers/owners who skip [O-ring](https://en.wikipedia.org/wiki/Space_Shuttle_Challenger_disaster#O-ring_concerns) warnings - -This doesn't change the fact that if your primary goal is aligned with the above [TL;DR](#tl-dr), you are what I consider a 'developer'. You are not a 'tester' (though sometimes you write tests) or a 'product manager' (though sometimes you have to fill knowledge gaps) or an accountant or a physicist or porpoises or even a manager (though you may have direct reports or lead a team of engineers)...the direct output of your work is *'working software that achieves the organization's goal'*. - -# The Organization's (Global) Goal - -For most organizations, even non-profit ones, what looks like a developer doing their job well [is to produce software that makes or receives money](https://dansilvestre.com/the-goal-eliyahu-goldratt/#:~:text=The%20goal%20of%20any%20organization,making%20money%20are%20non%2Dproductive.), or in some indirect but immediate way contribute to that end. Code can be many things, compilable or interpreted, imperative or declarative (or functional, etc.), sometimes even supportive of others doing those things (such as in the case of infrastructure-as-code for the release process). But never forget that: - -> The Goal of every organization is to make money, therefore so is yours - -For any given profitable startup in ["zoos next to the dragon sanctuary and unicorn exhibit"](https://www.cbr.com/cowboy-bebop-ein-origin-netflix/), or even run-of-the-mill enterprise Fortune 100 that can retain long-term talent (equally rare), revenue-as-code isn't hard to argue about. But for NPOs, let me walk you through it. - - - -Two sentient beings that are moving towards a common goal need to work together. To register as a non-profit, at least in most countries that allow for it, you need a 'board'...a group of individuals that can agree on a charter...and a charter, which includes goals. Non-profits either burn out on an all-volunteer model or hire/retain qualified [enough] individuals to drive and achieve the charter mission, or iterate on what should-be/is achievable at any given phase of the NPO. If an NPO is truly worth existing for any matter of time, money comes into play, either in management thereof or in order to compensate for said expertise and efforts. To generate revenue in any venture related to the software industry, at some point, software engineers are required. Even if they are volunteer, the non-profit organization needs to generate streams of revenue, for philanthropic targets, and sometimes to support the organization's core function to do so. NPOs need to 'make money' to survive, either to funnel to and justify their altruisms or/and to do that via skilled and focused professionals. - -# Developer-adjacent Likenesses - -If you're still not convinced that I've laid out a Goal, Approach, Outcomes and Tactics that sufficiently clarify what a 'developer' is uniquely, let me try a visual I've been working on called "A Dev Does as a Dev Is": - - - -In essence, think honestly about where you most feel comfortable. If that's not writing code with a clear mission and autonomy to implement well-understood requirements in some type of code, then you're not a programmer (disambiguation of 'developer'). If you are more on the problem-solving side, which is perfectly fair, you're still a dev, and code is a material part of your day-to-day. - -If you spend most of your time trying to avoid situations where you have to "resort to code", then you're not what I'd called a 'developer'. Not that a 'good developer' will rush to code any chance they get (this would also be an anti-pattern), but that the outputs of a 'developer' are primarily measured by *working software that achieves their organization's goal'*. - -NOTE: if you are a non-hashtag 'DevOps' person, you may be a half dev (like SRE time-spend) and half everything else that your organization needs. In this you are a special snowflake and worth every penny your company is (or should be) well-paying for you, since you think holistically about two sides of your work and of others' work. - -This is my blog and I can say whatever I want. If you don't like it, feel free to hit me up on [Twitter](https://twitter.com/paulsbruce) or [LinkedIn](https://www.linkedin.com/in/paulsbruce), quote and hate me, or better, leave useful comments that further this dialog. diff --git a/_posts/2022-03-09-verification-validation-volition.markdown b/_posts/2022-03-09-verification-validation-volition.markdown deleted file mode 100644 index ec0a3a9..0000000 --- a/_posts/2022-03-09-verification-validation-volition.markdown +++ /dev/null @@ -1,29 +0,0 @@ ---- -layout: post -title: "3Vs to Transform Testing - Verification, Validation, Volition" -date: 2022-03-09T09:13:41.000Z -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: - - testing - - devops - - teams - - engineering -tags: [] -author: - login: pbruce - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: /blog/2022/03/verification-validation-volition/ ---- - -NOTE: This is a placeholder article to house slides, comments, and notes related to - my presentation for [TSQA 2022 Wild, Wild Test](https://tsqa.org/schedule-of-events-tsqa-2022-conference) on 2021-03-09. - -- [Slides on Google](https://docs.google.com/presentation/d/1ikKk7i9iuvvYV0Leb0k1xi9WNWzVdaZYkJJLxp71CYo/edit?usp=sharing) - -Once the video is made available, I will link to it here. diff --git a/_posts/2022-03-25-telemetry-to-transform-testing.markdown b/_posts/2022-03-25-telemetry-to-transform-testing.markdown deleted file mode 100644 index 1def8da..0000000 --- a/_posts/2022-03-25-telemetry-to-transform-testing.markdown +++ /dev/null @@ -1,58 +0,0 @@ ---- -layout: post -title: "Telemetry to Transform Testing" -date: 2022-03-25T19:13:41.000Z -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: - - testing - - devops - - kubernetes - - "Cloud-native" -tags: [] -author: - login: pbruce - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: /blog/2022/03/telemetry-to-transform-testing ---- - -[Sign up to access the full broadcast](https://www.skilupdays.io/cicd-22/agenda/session/873478) - -[embed]https://youtu.be/Ju168RTf8dc[/embed]
- - - -# Session Title - -Transform Your Continuous Testing with (Open)Telemetry - -# Session Description - -Except for instrumented unit testing, it's often really hard to know what's exactly - what's going wrong when your tests fail, especially when our systems - are now highly distributed and involved multiple APIs, micro-frontends, and 3rd-party - services. Versioning across these dependencies and complex rollout processes also - further obfuscate what the heck is really going wrong when your tests fail. - -Enter "telemetry", and specifically OpenTelemetry. Technology that emits contextual - and timeseries-ready data about what's going on dramatically improve everyone's - ability to isolate, diagnose, and resolve issues quickly. - -BOTH systems AND tests that share modern, distributed context such as OpenTelemetry - span and baggage details transform testing into more precise and actionable feedback. -Come learn about how to inject context back into your work in this session. - -# Three Key Takeaways - -* How additional context dramatically improves actionable outcomes of testing -* How OpenTelementry applies to both software systems AND testing processes -* How to get started using OpenTelemetry in your code bases - -# Speaker Bio - -Paul Bruce is a passionate technologist, helping to transform enterprise software teams and delivery practices. He chairs o11yfest (May 9-12), volunteers locally, skateboards, and co-organizes DevOpsDays Boston and the Boston DevOps community. His technical research wheelhouse includes cloud management, high availability service architecture, API design and experience, continuous testing at scale, and organizational learning frameworks. He writes, listens, and teaches about software delivery patterns in enterprises and key industries around the world. Oh, and he’s hiring devs for his incubation engineering team! You can read more at: https://paulsbruce.io diff --git a/_posts/2022-06-20-opentelemetry-community-day-austin.markdown b/_posts/2022-06-20-opentelemetry-community-day-austin.markdown deleted file mode 100644 index 2425cf2..0000000 --- a/_posts/2022-06-20-opentelemetry-community-day-austin.markdown +++ /dev/null @@ -1,260 +0,0 @@ ---- -layout: post -title: "OpenTelemetry Community Day Austin 2022" -date: 2022-06-21T21:13:41.000Z -type: post -parent_id: '0' -published: true -password: '' -status: publish -categories: - - "OpenTelemetry" - - devops - - kubernetes - - CNCF - - "Cloud-native" -tags: [] -author: - login: pbruce - display_name: Paul Bruce - first_name: Paul - last_name: Bruce -permalink: /blog/2022/06/opentelemetry-community-day-austin-2022 ---- - -Preface: this blog post is just my travel log, personal reflections, and thoughts - from my time conversing with other community members at [OpenTelemetry Community Day - in Austin on June 20th, 2022](https://events.linuxfoundation.org/open-telemetry-community-day/). If any corrections or retractions need be made, [let me know](http://localhost:4000/contact/) and I'll be happy to do so! - - - -- [TL;DR - One Community Day Is Not Enough!](#tldr---one-community-day-is-not-enough) -- [What is OpenTelemetry in Three Sentences](#what-is-opentelemetry-in-three-sentences) -- [Why OpenTelemetry Community Rocks](#why-opentelemetry-community-rocks) -- [Session Takeaways](#session-takeaways) -- [After-thoughts from Austin back to Boston](#after-thoughts-from-austin-back-to-boston) -- [Summary](#summary) - -# TL;DR - One Community Day Is Not Enough! - -- OTel adoption, use cases, and its community continue to grow rapidly -- Those that could show up, did, and it was worth the trip -- Need more consumer uses/lessons presentations, but... -- ...the un-conference afternoon sessions were fantastic! -- The project NEEDS more contributors, to everything really -- Regional and regular meetups on OTel will carry the community forward - -# What is OpenTelemetry in Three Sentences - -"OpenTelemetry is a collection of tools, APIs, and SDKs. Use it to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) to help you analyze your software’s performance and behavior. OpenTelemetry is generally available across several languages and is suitable for use." - [opentelemetry.io](https://opentelemetry.io) - -The above three sentences are a great start, but of course there's more to it than that: - -- It provides a ***vendor-agnostic model for emitting traces and metrics*** (soon logs and profiling data) from systems -- Once implemented, it provides a staggering and necessary amount of backward and forward compatibility to not only its own componentry, but to a plethora of: - - back-ends (a.k.a. DIY or hosted storage and insights solutions - - processing filters (pipelines) - - languages / runtimes / frameworks to encourage visibility into new and legacy systems - - telemetry transformations -- It is a project in the CNCF that as of [August 26, 2021](https://www.cncf.io/blog/2021/08/26/opentelemetry-becomes-a-cncf-incubating-project/) is Incubating (next is graduated) -- It includes auto-instrumentation packages for top popular runtimes including (but not limited to): - - [Java](https://opentelemetry.io/docs/instrumentation/java/automatic/) - - [Python](https://opentelemetry.io/docs/instrumentation/python/automatic/) - - [Ruby](https://opentelemetry.io/docs/instrumentation/ruby/automatic/) - - [.NET](https://opentelemetry.io/docs/instrumentation/net/automatic/) - - [Node.js](https://opentelemetry.io/docs/instrumentation/js/getting-started/nodejs/#instrumentation-modules) -- Its many parts are maintained by many really fantastic individuals, some working for/with vendors, others working as consumers and community contributors -- As its key componentries intersect many languages and runtimes, various SDKs are a changing tapestry of readiness [statuses](https://opentelemetry.io/status/) - - - - -# Why OpenTelemetry Community Rocks - -In 2018-2019 I had a number of conversations ([re](https://www.youtube.com/watch?v=1O6hO8YLDwA)) with contributors to the OpenCensus and OpenTracing projects as they were realizing they needed to combine efforts into one project, OpenTelemetry. Unlike other online and project-driven "communities" I've experienced in the past, there was a sense of vendor-neutrality (though there's plenty of vendors in the observability space) and that this was an idea whose time had come. - -Since then, my contribution has been to cultivate an inclusive and non-insular community of people that want to learn, share, and (maybe even) contribute back to the OpenTelemetry project. For three consecutive years, I've been organizing [o11yfest](https://o11yfest.org), an observability-focused and community-driven event. Each year, [we donate equal or greater than the operating budget](/blog/2021/12/accounting-for-privilege/) of the conference to some [really good causes](https://o11yfest.org/sponsor#contributor-sponsorship-details). It is purposely NOT part of the CNCF, as we want it to remain independent from that financial machine. - -We also encourage contribution from everyone, not just voices from the CNCF and OpenTelemetry project, and put a fair bit of effort into [screening out vendor/sales pitches](https://o11yfest.org/sponsor#call-for-proposals) and blah blahs. This year (2022) in May, we saw more attendee engagement than the past two years, and even had a number of people submit ["pre-action" videos](https://o11yfest.org/2022/preaction) since all of our pre-recorded talks were available before the conference. - -Community is what you make it. So I'm doing my part, what I can, and looking forward to encouraging others to do so as well. In the next coming months, we'll be partnering with regional OpenTelemetry and observability meetup groups to double-down on contributing (hopefully and specifically about helping OpenTelemetry with their documentation needs). - -# Session Takeaways - -## Keynote: State of the OpenTelemetry Community - -Alolita started out with some project milestones and history. - - - -- Metrics delayed b/c ... priorities and proper core -- A variable landscape of readiness, with logs on the horizon -- OTel is growing! - how "community" is measured, only contributions? - - - -- "Independent" contribution group adds up -- "Compliance tests" (re: prometheus collaborations) -- Semantic Conventions, EU Feedback, Client Telemetry (front and back end signals) -- Client Instrumentation, Agent Management, and Profiling (re: [Ryan Perry, Pyroscope.io](https://pyroscope.io)!) -- So [maybe too] many ways to get involved; priorities and batch-fit to bandwidth - - Running OpenTelemetry meetups and publishing to the otel site - -## Community Updates - -OTel Comms SIG lead Austin Parker shared additional/complimentary info and thoughts - about the community at large - -- It's been to years since the last (virtual) OTel Community Day, almost all new faces -- Growth! 2x contributors, 3x contributions -- "Public Adopters"; and info shared by "vendor-partners" -- "Documentation needs help!" - CNCF Slack #otel-comms -- CNCF now accepting ["OpenTelemetry Ambassadors"](https://www.cncf.io/people/ambassadors/) - - - -## Unconference Session Topics - -## Maintainers Panel - - - -- challenges - - ??? (something I missed, need to coordinate with others) - - [contrib repo](https://github.com/open-telemetry/opentelemetry-collector-contrib): how is it maintained, patches applied, etc. - - single owners for specific instrumentation libraries - - not enough engineering hours - - note: how would we incentivize contributions (not just usage and issues) -- Q: how do you relate [slack](https://cloud-native.slack.com/archives/C03JFUAJXT4/p1655738403456849) - - Dan: bubble of "enlightened"...??? (will reach out to Dan in CNCF for actual point made) - - Jack: if your biz has dependencies on an OSS thing, it's strategic to have eng knowledge via contrib - - Aaron: start small, even 10% of time is a big ask; easier to "made a fix, can I release?" - - Amir: I don't do it for the vendor...infers trust over "planned work vs. bandwidth" -- Q: Biggest challenge getting library authors to instrument their libraries natively...? [slack](https://cloud-native.slack.com/archives/C03JFUAJXT4/p1655738729865949) - - Amir: implementing natively right now may be...complicated...as some things aren't yet GA'd - - Ted (Young): not quite yet, need things to be stable - - Dan: wouldn't want to ask someone to make effort only to break it for them later -- NOTE (Austin): need help with OTel Collector and output being 'certified' in supply-chain -- Q: What have been the biggest challenges with implementing the API and SDK across the supported languages in a standardized way? How much does the SDK configuration feel 'native' to the language? - - Dan: forcing things like naming conventions - - Jack: the spec is more ergonomic guidance -- Q (Audience): [observability at Adobe] For other SIGs, planning to add auto-instrumentation as standard across - - Jack: (Trask? should have been here, Java auto-instrumentation) - - Dan: When APIs aren't stable yet, any change means changing lots of dependencies - - Dan: JS backwards-compat esp. around Node packages has been tricky; we're working on it - - Aaron: it's tough for Go, can't patch just wrap things; it's a monumental task, maybe wrapping is the best - - Dan: the Operator DOES do some automatic injection...Node, Python, Java -- Q: What's the recommendation on PULL/PUSH based methods, and how does OpenMetrics fit in? - - Aaron: if you have something that's working, use that. OTel collector is a bridge between app and backends. - - Dan: challenge the assumption that Pull-based "won", common deployment is to chain collectors - - Jack: The collector is your friend, it's a translator and an enricher of telemetry - - Dan: about interoperability, we've been working a lot with Prometheus team closely -- Q: If you *did have all the contributor bandwidth you could desire, how would it be put to best use? - - Dan: seeking and placing diversity of skills and expertise that isn’t already in the project - - Amir on ease-of-adoption, which I think is already starting to be changed via the End-user Feedback group. - -## OpenTelemetry and Service Meshes - - - -After beers with the Aspecto team on Sunday at [Easy Tiger](https://www.easytigerusa.com/), it was clear that these guys are on a mission to put a [much needed] dent in our universe. On the walk back, Eran talked about how being a founder in this space requires an extreme amount of focus, which IMO they have and are demonstrating on multiple fronts. - -Without betraying any particular details, the conversation revolved around themes like "how contributing to OTel isn't at all about what your employer pays you to do", "how much 'magic' people expect without doing anything", regulated industry constraints with adopting recent/moving/early OTel components, and how much interest there is in the thriving observability community in Tel Aviv. - -The short is, for a Boston boy like me, these guys know their shit and work hard as fuck. - -Michael, who coincidentally [spoke about Malabi at o11yfest in Ma](https://o11yfest.org/speakers/michael-haberman)y, made the following great points: - -- just because you turn on traces doesn't mean that you are doing tracing -- traces without shared context are just more noise to signal ratio -- the context added via mesh about service paths is critical to understanding versioned routing -- deployed in a service mesh, the mesh IS part of the app, no just wrapping -- trace "brokenness" includes no root span; gaps in traces indicate a critical lack of context propagation -- bad news: - - increase cost (more spans) - - what about selective/adaptive propagation? - - head sampling (only percentage, thus some waste) - - based on root from client? - lots of work - - Configuration (Envoy changes needed to export OTLP) - - Maybe next year, we'll have it all solved -- what's interesting about meshes - - many companies he talks to who are using meshes, are also using an ecosystem of additional functionality - - just like plugins, they can cause issues when combined together - - which means we NEED better telemetry and tracing - - - -## Personal: Hiring Break - -I then took time to meet with a late-stage candidate for Product Manager of our Incubation Engineering group. -[My group](https://www.tricentis.com/company/careers/) got to an offer acceptance, so that was a really nice lift going in to the afternoon sessions. - - - -## Lunch and Networking - -Since the event was somewhere between 50 and 75 attendees, I found it very easy to be cycling back through the clusters of folks who I had already chatted up, all within about 45 mins. This was actually nice as a forcing function to really strike up or contribute to meaningful dialog, since there was very little room to wander. - - - -## Debugging OpenTelemetry - -Sadly, having had to take a call, I the first part of this lightning talk from Ted Young. However, subsequent comments from other attendees confirmed what everyone already knows which is Ted is awesome and 80% of the time he speaks it is useful...all the time. [His 2021 o11yfest presy about The Value of Design in OpenTelemetry](https://o11yfest.org/speakers/ted-young) stands as true amidst a year of major efforts to GA as it did back then. - -However, I spent the better part of the afternoon in un-conference discussions - with Ted and other attendees about Tracing and Testing topics, so while I missed - his most recent incarnation of coolness, and his body of work speaks for itself. - -## Breakout Sessions - -### eBPF and auto-instrumentation - -The best part of the day, seriously, but as open and engaged conversations go, I - was more invested in the people in the room than taking notes on my laptop. - -The short is that most people in the room needed a simple description of what [eBPF](https://ebpf.io/) is - and very few people who were versed on the topic. Key taking points are: - -- lots of buzzword interest in "eBPF" as an emerging topic -- many people looking for a magic solve to "observability" via auto-instrumentation -- how does eBPF improve the auto-instrumentation motion -- auto-instrumentation...does it help teams adopt better telemetry or impede intentional thought -- how auto-instrumentation is a stepping stone to make teams want more...and do more to get it - -Fortunately, we had an amazing person (Libby) who volunteered to capture topics on - the sketch board. - - - -### Sessions I didn't attend (but would want to) - -Sadly, I cannot physically be in two rooms at once. Hopefully there was a recording - of this discussion, but if not and someone has notes, I would LOVE to link to them - from this post here. - -- Intermediate OpenTelemetry and Signal Correlation -- Continuous Profiling -- OpenTelemetry Collector - - -### Tracing / Open Telemetry for Testing - -Again, being invested in the conversation and capturing meaningful topics was more - important than taking copious notes. As such, I did take the cue from Libby and - make sure that at least some of the key points of the discussion were documented: - - - -# Summary - -Worth the trip. Lots going on in this space. Needs contributors, not simply adopters. - Get involved. Reach out to me or any of the visible core maintainers. Expect growth. - Come be part of the community as it thrives. - - diff --git a/assets/css/main.scss b/assets/css/main.scss index cfa2432..1a75c6c 100755 --- a/assets/css/main.scss +++ b/assets/css/main.scss @@ -88,6 +88,7 @@ h1,h2,h3,h4,h5,h6 { .initial-content { padding-top: 1em; + padding-bottom: 2em; z-index: 3; }