My ramblings on the tech industry, life, and random experiences.
Feb 27, 2017
Warning: This is going to get oddly philosophical partway through, and some of the material in this post might be considered offensive by some. Apologies in advance on both accounts.
Anyone that knows me personally knows that the amount of irony surrounding the fact that I work in Ed Tech is not small. In school I did pretty poorly once I had to start turning in homework. I'd find any of a billion other hobbies to work on or things to do instead of doing homework. And even more ironic is the fact that my favorite leisure reading material is History, because it was my least favorite subject in school. My brother got his masters in History, and that always baffled me because history class bored me to tears. Reflecting on it though, I think I may have the answer as to why that was.
In High Schools in the southeast, it's not uncommon for the football or baseball coach to have to teach some subject to keep his position within the school system. These coaches are typically saddled with the non-vital subjects such as History, Social Studies or Geography. By non-vital, I mean that a student can typically graduate high school without having done too well in these courses. The common theme with my coach-teachers was that they had very little interest in doing a good job at teaching; it was an obligatory task that had to get done so that they could go back out to the field and win games for our school. The resulting effect of this was that the subject material was delivered in a very dry, uninteresting fashion. Students just had to memorize dates, names, and places, and they would get a passing grade. This is a huge contrast to my chemistry teacher, Mr. Norton, who on the first day of class walked into the room with a bucket of liquid nitrogen, faked a stumble, and dumped it on the ground so that we could see how it would react with its environment. Mr. Norton was passionate as hell about the subject, and he loved teaching. I did very well in the sciences in high school as a result of the fact that the teachers took the time to make it interesting. Even at that age, I knew something needed to be done about the way we deliver education to kids. Not everyone has the same interests, nor do they learn in exactly the same way as other students. It is not wrong to learn more easily in a fashion that is different from someone else. In my humble opinion, we NEED people that learn differently. I feel as though we're losing out on lots of opportunities for students to achieve greatness by trying to squish them into the same mold.
It would never be admitted, but it's possible that part of the reason I was allowed to graduate from high school may have been that I found a way to steal internet access from a nearby University for our high school to use. The assistant principal, who I am still in contact with and is a terrific guy, was also tech savvy, and took a real fancy to the internet. He and I spent many hours in the computer lab browsing usenet groups, chatting with people and programming on muds, and playing Netrek. We were showing teachers how to use Email and setting them up with accounts in 1990, long before anyone in our region had ever heard of the Internet or email. I was 'learning', but not necessarily what was being taught.
Now this is where this post takes a turn. It gets dark from here. And if you believe that evolution is a gigantic hoax perpetrated by scientists who want to push their sinister liberal agendas, I would recommend stopping at this point, and picking up a good book.
Anyone familiar with the Fermi Paradox or the concept of The Great Filter has likely gone through their own mental exercises regarding what brings life on other planets (which statistically should exist) from reaching the point where they can be 'heard from'. To summarize, The Great Filter is 'whatever it is' that prevents this from happening. This boils down to a few possibilities:
But I propose that there is an additional possibility, which is that at a certain point, species stop evolving due to the lack of necessity for natural selection to run its course - once the species have figured out how to maintain themselves with minimal effort. My theory is that this may lead to the inability to create technology necessary for space travel, feeding overpopulated planets, or being able to establish world peace.
In the caveman days, any mutation involving increase in cranial capacity could've meant a better chance at survival. It was vital to understand the environment around you, and to be able to predict when and how you were going to arrive at your next meal. This took some level of smarts. Agriculture likely has put an end to this cycle. We can now survive, completely independent of our mental capacity, just by virtue of the fact that we have a pair of legs and enough command of the local language to be able to order from a fast food menu. My theory is that for us to be able to travel to or even communicate with species of distant planets would require knowledge and understanding that perhaps our current brains are not - and may never be - capable of obtaining. NASA has just discovered 7 planets that could potentially possess life, even intelligent life, only 40 light years away. Still though, that's 40 years, and we would need the technology that would allow a spacecraft to achieve the speed of light. We are *ahem* a bit of a ways off.
I've put these gloomy ideas before you not to indicate that we should all give up, or convey that we're doomed ( which we are, have a great day! ), but to say that in the mean time, given our stunted development, only education can push us forward. Only education will provide for us the ability to feed a planet with 10 billion people, or to help us to slow down or reverse the damage we've done to our environment.
Our current president wants to bring back all of the manufacturing jobs, and make America Great Again. I still maintain that the orange man in the white house has good intentions, but he may be going about things The Wrong Way. Bringing back low level and manufacturing jobs may be a lost cause, given all of the momentum behind robotics and AI. These people may be returning to a workforce that is dissipating as quickly as it is created due to the tremendous advances in automation. If we double down on education though, we can provide these individuals with the skills necessary to work in this new world of technology. This may seem far fetched, but think about fertilizer for a moment. In the mid 1800s, Jean Baptiste Boussingault was experimenting with nitrogen and its effects on topsoil for farming. To the farmers of his day, this must have seemed something like a cross between black magic and rocket science. Today, millions of farmers the world over use some sort of synthetic fertilizer that enriches the soil with nitrogen via processes that have resulted from this initial experimentation, yet we don't consider anyone who knows how to mix fertilizer with soil a scientist or even a highly skilled laborer. The autonomous robots that will soon be harvesting the majority of our crops have analog components and mechanical parts that will need to be serviced; many tasks that will require some training, but may range from being as simple to changing a tire or swapping out a faulty set of GPUs, to programming the navigation code that allows the robot to move about and do its job.
With the advent of big data, advanced learning platforms, and all of the behavioral analysis that can be done at the intersection thereof, we can educate future generations on these technologies effectively and efficiently.
And who knows, maybe we can bring about world peace.
April 14, 2016
Not to wax too cynical on the subject, but I'm often cautious when I hear the term 'full stack developer' or see it on a resume.
Now, I'm fully aware that the world has evolved beyond having to schlep gear to a colo and register your own AS number in order to be a presence on the Internet, and I'm fully aware of the amount of mileage that you can get out of a pure cloud implementation without the need to really support ( or even understand much of ) the underlying infrastructure. However at some point, if you're in the SaaS business, you'll need to scale up. And scaling up means understanding more of the stack than the pile of third party libraries/SDKs/frameworks that make up your 'front end' and your 'back end'.
So let's take a look at what makes the mythical 'two halves' of our SaaS platform tick.
The Front End
I overheard a conversation a few years ago between two front end developers in which one asked the other which 'framework' his team at YouTube was using for their front end. When the YouTube front end engineer replied that they "didn't use one" and that their team lead required that all of their JS code fit into a single TCP packet, the other stood staring as if some mechanical malfunction had triggered an error inside of his exception handler. This made me smile.
I asked a full stack developer a couple of years ago during an interview what 'Transfer-Encoding: chunked' meant in an HTTP header and he couldn't even produce a guess. My point is not to say that he wasn't a particularly good developer, but shouldn't the networking stack be considered part of the 'full stack', considering it is the artery that delivers the content to the consumer?
What also worries me is when a front end developer insists that the back end do work that the front end can easily do, such as sorting, transforming, or merging data when having the back end do the work could involve creation of temporary tables for buffering sorts, or creating extra overhead for manipulating the data when the data transforms are only necessary for a particular view of the data. Client CPU is essentially free. Unless you are going to impact user experience - use it. Keep your API endpoints lean, fast, and efficient as possible, or you'll be paying the price.
A while back I saw a tech blog post where a team had used a pool of async workers to dynamically generate images for icons which were essentially colored boxes with a couple of text characters over top of them. Figuring out why this seemed like a better option than just storing the characters and a hex string for color for each object and having the client browser visualize the icon will require that I take a trip to the pub and marinate my brain in a few gin and tonics.
The Back End
Back here is where I generally get irritated.
There's a lot more to building a scalable API than knowing a high level language and using a popular ORM to talk to a database, but it's a start. Picking up a good understanding of cache engines, service oriented architectures ( and knowing when/why/where to use them ), load balancers, storage systems, IP and layer 2 networking will get you a long way as well. Some other concepts I find a lot of full stack developers fall short in knowing include:
I/O. Data store I/O is a killer. What may very well be the first rule of thumb in designing scalable back ends is: if at all possible, do not touch disk. At the very least, fetching data from a physical disk or SSD should by far be the exception, and not the rule. The slowest point in your system where bottlenecks are most likely, is your persistence store. There are wonderful memory based data stores to be used for cache, so use them. And when a file/document/object/rhinocerous/whatever is in cache, and an update to it occurs, write the update to cache, don't rely on the cache expiration to update what's there. Otherwise you wind up with a stampede of app servers all trying to populate the missing data at the same time, and your whole platform can go straight to hell in a hand basket.
Understand your app server. You should know intimately the pros and cons to having a pool of single threaded app servers vs a multithreaded app server.
If you can do it, process asynchronously. I'm not talking about asynchronous app servers such as NodeJS, but rather the concept of data processing via API calls. Unless you absolutely need the processed data in the payload in real time, you should probably consider dispatching it to a farm of workers and using a message bus or similar concept to send notifications when the processing is done. One of my current favorites is RabbitMQ + WebStomp. Free up your app server threads to handle more requests. Sure your user_details API endpoint only takes 4ms on average to respond, but if the client request for it has been dispatched to one of your single threaded app servers which is currently processing an API call which takes 2 seconds, the browser that is requesting that user data is going to wait, and the effective response time for the user_details call will be wahtever's left of the 2 second API call plus 4ms. NewRelic uses a simple header injection technique to measure request queueing. If you're a customer ( and I can't see why you wouldn't be ), then I highly recommend using it.
Know how to pick the right database, and understand extremely well how your app will need to talk to it. Throwing a NoSQL-based platform into the mix without being able to answer why it's a good fit is a good way to get your resume placed at the bottom of the paperweight stack. Is your app write-heavy? Read-heavy? These important details should dictate what kind of database you use, how to layer out your storage devices, logical volumes etc, what types of indexes you will need, and which types of systems to install to begin with. If you find yourself creating indexes on tables because your ORM decided to generate queries a particular way, you may be fighting a losing battle. You, the app developer, should be dictating how your app talks to the database and determining what indexes you will need throughout your application.
The Other Stuff
If you are truly a full stack developer, in addition to understanding storage, networking protocols, application layer protocols, system tuning ( which I didn't even touch on, but may rant about in the near future ), and all of the fun stuff that is required to conquer the front end and back end, you're probably also familiar with NTP, SMTP, and a flavor or two of monitoring systems, SNMP or otherwise. You're also likely comfortable using a network analyzer to debug communication issues and identify bottlenecks.
The idea is not to say that anyone who claims to be a full stack developer needs to have built entire SaaS platform from cabling wires to building out massive storage systems, but one should fully understand how an application talks over a network, and when and why it talks to its data store, in order to be able to construct scalable systems and brand oneself 'full stack'.
September 13, 2015
I don't often wax entrepreneurial, mostly for distaste for the word and partly because I see so many people offering their advice that it seems like the room is a bit too noisy for a conversation to be had. However, this particular topic I've not seen touched on, and seems to me to be important enough to write on.
In this day and age when everyone is an entrepreneur and startup founder ( or more gag-inducing a 'serial entrepreneur' ), it's odd to me that so many articles I see posted on LinkedIn are telling people how Steve Jobs ran his meetings, or how tech superstar John Smith runs his middle management team to achieve X, Y, and more importantly, Z. Seems like everywhere people are offering up or sharing what they think the blueprint is to being a succesful startup owner, and too often ( in my humble opinion.. but hey, this is my blog, so you're obviously going to be getting my slant on things ) it's by example.
There are two main types of perils here: The 'Follow Ideology X' problem, and the 'Do things like Y does them to achieve wild success' problem.
Firstly, the 'Do Things Like Y does' problem: Thinking that you can replicate what Steve Jobs did as a successful CEO may not work at all with your personality type, the chemistry of your team, the goals of your company or its culture. Chances are, whatever it is that Steve Jobs did to manage his meetings wasn't something he found in a book he read by Guy Kawasaki or any other piece of tech literature. He likely tried a few different approaches, figured out what bits worked the best, and left the ones that didn't behind. Isn't that what being an entrepreneur is about? If you don't experiment, and figure out what works best for you and your team, you may be heading down the wrong path, regardless of how well it worked for Him. So stop following, and go out and do what works best, and if you succeed at it, take pride in the fact that you did it your own way.
Secondly, let's think about the Follow Ideaology X problem: What I mean is, as with anything, following some concept or suggestion to an extreme is dangerous. I know a wildly successful mobile application company which has followed The Lean Startup to the point where it's now hurting them. Not that Eric Ries doesn't have a plethora of valid points and great ideas in his book, but there comes a time when a company needs to realize that it's not a startup anymore, and 'doing one thing really well' isn't going to work when you're in a crowded space and many of your competitors are doing that one thing really well too. Worse yet, they've found something that's supplemental to what they're doing that leads them into breaking new ground, or an adjacent space where they can execute on a successful revenue strategy. I'm of course not saying that you as an entrepeneur shouldn't read books written by individuals that have blazed the trails before you - what I *am* saying is these books should be reference materials to draw ideas from, or to help you form the types of experiments that will allow you to find your own success and make it yours.
August 15, 2015
I suppose it happens in more than a few peoples' lives but despite my humble origins, asking a passerby on the street for change wasn't a regular occurrence.
At the age of 17, with no money and having barely made it through high school, my dream of going to art college was obviously out of the question. And as if by chance, an Air Force recruiter had begun pestering me. I did have a secret desire of becoming a pilot for most of my youth, and this was good enough ammunition for the recruiter to convince me to take the Air Force placement test (ASVAB) which I did surprisingly well on. After that, all I needed to do was to pass the physical exam and get sworn in, and I was on my way. My father was somewhat pleased with this, having been in the Marines himself and we both agreed that it was a reasonable route for me, given the situation I was in and the potential career development that could come out of it.
This was the summer of 1992. Having worked several odd jobs ( starting at the age of 13 ) I was prepared to work and scrape to get by. I had alread had a roommate and lived on my own, but when times got really hard, my father reluctantly let me move back home for long enough to know that I needed to find another way out into the big brave world, so the recruiter's timing seemed fortuitous.
I suppose I was like a lot of other kids my age that didn't have the way or means into university, with one small exception. I had already been introduced to and had fallen madly in love with the Internet. Not what you know and experience today, but a pre-web, pre-e-commerce internet driven largely by curiosity, DARPA and some smatterings of grant money. A good friend ( and to this date one of the coolest people I've ever known ) introduced me to it by explaining it to me on a road trip we had undertaken to deliver him back to the university one winter. I was instantly mesmerized by the possibilities, and knew immediately that this would shape our collective future. I just had no idea what exactly that future looked like. Regardless, I proceeded to spend as much time as I could at his university ( Duke ) and conveniently enough, we looked a lot alike (we'd been mistaken to be brothers on more than one occasion) and this little fact would grant me access to all of the compute clusters at Duke where I became reacquainted with computers after a several year hiatus.
I had first been introduced to computers while in a special program at the elementary school I had attended in Hiawassee, Georgia where we were given access to a TRS-80 ( lovingly referred to by us old schoolers as the Trash Eighty ) where we had been taught the BASIC programming language and were more or less set free to see what we could do with ourselves. As a long time Dungeons and Dragons aficionado, text based adventure games fascinated me, so I wrote a few of those. When Jason sat me down in front of a DEC Ultrix machine at Duke, I realized that these computer things had evolved quite a long way. Consequently, it was a year or more later when I was shown a Windows 3.11 machine and it felt so uncomfortably primitive that I couldn't imagine that there would be any use for one at all, and was dumbfounded by the fact that they didn't incorporate the IP stack in their OS. Even the Mac seemed to be on its way and leaps and bounds ahead of the Windows platform with regards to networking in general.
So by the time I approached the medic at the recruiting location in Charlotte, North Carolina where I was then supposed to be sworn in and swept off to basic training, I already knew my way around a unix shell, some basic IP networking, some scripting, and a bit about the workings of the Internet. Yet I had absolutely no idea that this information would ever become valuable, and had been ridiculed a fair amount for being so caught up in such nonsense to begin with. So when the recruiter asked me that fate-laden question, 'Are you allergic to anything?' it hadn't dawned on me that despite all of the recruiter's recommendations about what to say in response to any of the physical examiner's questions ( "If he asks you if you've ever sneezed, you say 'no'" ), that my being allergic to wool would have any impact on my ability to join the Air Force and blow up people and things with expensive weaponry. Yet impactful it was, as his immediate response to me was 'Alright. You're out.' He made a quick gesture for the next kid in line to approach his podium, and I managed to throw in a question, which likely sounded something like, "I'm sorry.. What?" To which he replied - and on this I'm a bit more certain of - "All of our shit is wool." Now, being able to rationalize this in my head had no affect on the fact that in just a few moments, I would find myself on the streets of Charlotte looking for a quarter to use for the payphone to call my father to pick me up. My father was furious, as I had finally been close to some semblance of a future, and had blown it with complete ignorance.
But Dad was angry for a reason, and although it makes perfect sense today, being upset at your son for not giving any indication of trying to do and be better befuddled me at the time. However, it was a good motivator for me to get out on my own again and figure something out. Two years later, at the age of 19, I would start my first company with a group of really bright kids around the same age as me, and although we would ultimately fail, the experience set the stage for the rest of my life.
January 6, 2015
I consider myself somewhat of a technologist, and it is true that technology is a passion of mine. When I came to Silicon Valley in October of 1995 from South Carolina, it was more than I ever could have dreamed of. I had no idea what was going on out here, and that there was a whole valley full of people as passionate as I was about this giant network that I'd fallen in love with. I'm not sure if I had even heard of Silicon Valley - I just knew that I had a job waiting for me working at an Internet Service Provider that had been far more successful than mine had been ( though granted, I had been trying to convince Carolinians that the Internet was going to be more exciting than re-runs of MASH ). Here, the whole place was buzzing with talk about new ideas, startups, new programming languages, routing protocols, and people trying to get their heads around the Internet and what it meant for our future. People weren't drawn to it because they saw the next get-rich-quick scheme, or because it was trendy to call themselves entrepreneurs. Hell, people rarely even used that word back then, even though we were all starting companies, or working in startups. In those days, a startup wasn't a guy who paid some overseas software shop to crank out an MVP to run on a couple of cloud instances hoping to be the next WhatsApp. We were in it because the technology itself was fascinating and we wanted to be part of it. If you wanted a serious web presence, you paid for rack space or a cage in one of the colo providers in MAE-WEST or the PAIX where you would schlep big expensive (yet elegant, beautiful, and reliable) Sun gear to host your site or service on. You had to really want it, and fully believe in what you were building. You bought and configured big pieces of networking gear, load balancers and storage systems to scale your environment out - and it was expensive. To host anything of significance you needed to understand BGP ( the backbone protocol of the Internet ) and have an ASN, an IPv4 block, and know a fair amount about power, cooling, and systems. The concept of a 'web developer' was very new, and most of them even knew how to build out systems, and a good bit about networking and lower level programming. You didn't call yourself a programmer or any sort of engineer if you didn't know a fair amount about most or all of these things.
Sure, I stay current. I still spend a lot of time keeping up to date on what programming practices or design methodologies are out there. I create projects for myself in new languages so that I can understand their approaches to solving age old problems and new ones. I can't say that I relate with their developer communities though. I can't subscribe to the flawed philosophy that a developer shouldn't have to know how an application is talking to his database, or the fine details of what goes on in the underlying system or storage cluster. Those people are like ticking time bombs for some company to hire to build out their platform. They'll get your prototype out the door at light speed, but put any traffic on it and prepare yourself for a bill the size of a Pirates of the Caribbean movie.
But worse than the brogrammers, I think it's the 'entrepreneurs' that bug me the most. The word feels so tainted now. A friend showed me a profile recently of a guy who claims to have started more than 100 successful companies during the span of a 20 year career or so. My guess is that his criteria for having 'started' a company is forming a cohesive vision for a product for a few minutes before lunch. And of course this individual had about as much technology under his belt as Barney Fife. So not only did he not start and run these alleged successful companies, in the couple of months he would've supposedly been with each of them, he wouldn't even have been able to build the products either.
In contrast to those golden days, the tech industry today seems to lie at this horrible intersection of the mysteriously entitled generation Y, the millenials, and the extremely cheap and available resources for getting a product to market that the cloud and inexpensive overseas outsourcing shops have created. The latter in and of itself isn't necessarily a bad thing, but it's allowed for the scene to be polluted with the same get rich quick types that ran out and became real estate agents during the real estate bubble. Of course there's nothing wrong with someone figuring out how to make money to support their family, but there's no soul in doing something purely for cash either. Sure your cousin Joe really took a shine to real estate. Now tell him that tomatoes have gone to 20 dollars a piece and see what he does to his front yard. The tech industry was never a place for those people - we were tinkerers and researchers at heart that would have been doing the exact same thing were there not even a promise of a paycheck - just because we wanted to. We were the kids that took every one of our electronic toys apart to see how they worked. We made blue box dialers out of parts we bought at Radio Shack and not because we could save money on making phone calls from payphones - we did it because we could and because we were hungry for whatever piece of technology we could get our hands on. For us, the Internet was this vast sea of geek mystery and we wanted to explore it all and be part of its amazing expansion. We did it because there was an inexhaustible quantity of information to be learned about a subject that was dear to us. We used archie and gopher to transfer open source software around and share knowledge. We snuck into computer labs at neighboring universities to get our hands on computers that we otherwise would have no access to. And there was altruism within the Internet community.
Those golden days are long gone, and it doesn't feel like there will be another wave of innovation that will bring us back to those magical times when such an earth shattering revolution of technology will be solely in the hands of those that love it for what it is. But then again, had it not gone the direction it did, I wouldn't be able to sit here and wax nostalgic about having been a part of what I feel was like witnessing the very birth of rock and roll. ░
December 8, 2014
It’s important to know a little bit about my life for this to make any sense. My wife is a 5 foot tall, beautiful brazilian woman. We’ve been married for about 14 years, and have vastly differing senses of humor. Most of the time, and yes, often when inappropriate, i’m trying to be funny or make people laugh. this works on her almost never. When it does, she replies with her subtle portuguese accent ‘you are so stupid.’ She told me a few years into our marriage after i’d heard that from her at least a dozen times that she means more like ‘silly’ but then gave me that look she does when she’s trying to figure out if i've caught on to her sarcasm.At any rate, it's important to have that bit of background to truly appreciate conversations like the one below. It's also important to know that she has an inordinate number of unique facial expressions, all of which can speak volumes at the right moment, and it also helps to paint the picture understanding that her default facial mode is an impossible combination of angry, and extremely cute.
And without further adieu, I give you one of the hundreds of reasons I wind up sleeping on the sofa:
The interesting prologue to this conversation is that Andrea had come up with this exact 'one shoe' idea almost a year prior. I, of course, had completely forgotten.
November 28, 2014
Another important thing to know about my wife is that she is extremely social. It's very hard to keep up with social engagements, partly because I tend to find out about them only hours (or minutes) in advance. On an average week, we have at most a couple of nights where we do not have dinner guests, or are not going out to dinner to meet up with someone.
What a lot of people don't know about me, is that by nature, I am an introvert. I socialize plenty (mostly for lack of an option) and do enjoy it for the most part, but I eventually need to recharge my batteries and spend some good quality time alone and in perfect silence, or listening to the calming rhythm of myself banging my head on solid objects. However, those blissful moments are very rare and are typically broken by conversations such as this one: