Subscribe: Jeremy's Thoughts
http://katzj.livejournal.com/data/rss/
Added By: Feedage Forager Feedage Grade B rated
Language: English
Tags:
back  bit  cloud  fedora  good  lot  new  nice  people  point  pretty  ride  thing  things  time  work  working  year 
Rate this Feed
Rate this feedRate this feedRate this feedRate this feedRate this feed
Rate this feed 1 starRate this feed 2 starRate this feed 3 starRate this feed 4 starRate this feed 5 star

Comments (0)

Feed Details and Statistics Feed Statistics
Preview: Jeremy's Thoughts

Jeremy's Thoughts



Jeremy's Thoughts - LiveJournal.com



Last Build Date: Tue, 10 Sep 2013 14:42:10 GMT

 



Build systems are the new black

Tue, 10 Sep 2013 14:42:10 GMT

So I got into a little bit of a twitter argument last night about distro packaging. Which actually wasn’t where I was trying to go (this time (image) ) One of the problems with twitter is that 140 characters can be hard sometimes. So let’s see if more characters help.

There is a big push afoot from a lot of people towards omnibus packaging. It seems especially prevalent in the world of things written in Ruby I suspect because most even “current” Linux distros are or were shipping some Ruby 1.8.x build up until very recently. And people want to take advantage of the newer language features.

I have a lot I could write on this topic. But I’m not going to today.

The main point I was trying to make is that in going down this path, people are putting together increasingly large build chains that have complex dependencies and take a long time. And so they’re starting to do things like caching of sources, looking at short-circuited builds and a whole lot of other related things. Which, incidentally, are all things that all of the Linux distributions ran into, fought with and figured out (admittedly, each in their own way) years ago. So rather than just rediscover these things and do it yet another way, there’s a ton of knowledge that could be gained from people that have built distribution build systems to make things better for the omnibus world.

So reach out to people that have worked on things like the Debian build system, the OpenSuSE build system, plague, koji and a slew of others. There are some tricks and a lot of tools which are just as valuable as when they started being used. Things like caching build roots and how to fingerprint them for changes, what ccache can and can’t be good for, how to store things reasonably in a lookaside to not depend on upstream repositories being down while you build, incremental builds vs not, ccache, …

Just because you think that building system RPM or DEB packages doesn’t meet the needs of your users doesn’t mean that you have to throw away all of the work and experience that has gone into the toolchains and build systems present there.

Originally published at Jeremy's Thoughts. Please leave any comments there.




One year with Stackdriver

Thu, 08 Aug 2013 14:35:16 GMT

A year ago this week, I started at Stackdriver.  As the first engineer with the company, there wasn’t anything when I started. We had some ideas, solid funding and a small core group to start figuring out our product and building it. Working out of our first “office” in a classroom at Northeastern Going in, I thought I had some idea of what I was doing and what I was in for. I mean, I’d been at later stage startups and heard the stories. I’d read the Eric Ries Lean Startup book. I had read all of the “blogs you’re supposed to read” and had them in my RSS reader so that I could soak up collective wisdom regularly. In some ways, I was right… in others, I was wrong. Here’s a non-exhaustive list of things that are sticking with me after a year. That said, if you’re reading this because you’re thinking about or planning on joining a startup at an early stage, there’s a good chance you’ll have different ones that are important for you Hiring is the single most important thing you can do. Good hires amplify everyone around them and make the team smarter and more effective. It’s pithy and everyone says it. But they say it because it’s true. Hiring takes way more time than you think it will. Your network (and your network’s network) is the best way to source candidates. You’ll post your positions on various job boards and you’ll very very very occasionally get lucky with it. Recruiters will come from everywhere promising great results but will take up even more time since their candidate flow is way higher. Another challenge with recruiters is that when you are before the launch of your product, it can be difficult to convey what you’re doing as well as exactly the type of person you are looking for to join the team. Fancy new technology is sexy and fun to work with. It also has a lot of problems you don’t have experience with. If you can minimize the number of these you have to deal with, you’ll be able to spend more time focusing on building the right product for the right problem It’s hard to spend too much time talking with your target customer. If you think that what you have (be it a mockup or a prototype or something else) is “ready” to show to them, you’ve probably waited too long and could have gotten feedback already and learned something Fail fast. Some things you try won’t work. Don’t continue to fixate on them and sink more time on them if they’re not coming together. If you’re building something that’s a B2B product, think of your beta as a chance to test selling in addition to the product. Sure, you’re not going to have them pay today but you do want to know that you’re building something that people will pay for when you’re ready to switch to that. Think iteratively. Once you start to hone in on a degree of product market fit, you’re going to discover that some of the things you built for prototyping/testing purposes don’t work. Don’t be afraid to replace them. But do so in a way that lets you regularly checkpoint the replacement to test that you are making things better. Grand rewrites are rarely as grand as you think and always take longer If you’re going to spend a lot of time on something product-side, focus more on getting the interfaces right than the implementation. So, for example, if you decide that something is going to have clear boundaries passing messages over a queue, then you can switch between RabbitMQ, SQS and others with minimal effort as you learn the constraints that actually matter for your implementation. Try to find the one piece of your product that immediately pops in a demo for most of your target users. This is one of those things where you’ll know what it is when you see it. And then use it as a hook to start drawing people in. Interactions with the customer don’t end when they sign up for your product. Continue to nurt[...]



The 2013 Assault on Mt Mitchell

Mon, 27 May 2013 18:00:32 GMT

As a kid growing up, one of the things I enjoyed doing was riding my bike.  In the woods, on the road, anywhere.  I even did some group rides at the time although I was on a mountain bike for them. And I remember hearing of some of the bigger rides in western North Carolina at that point… Bridge to Bridge and the Assault on Mt Mitchell, notably.  So when I really started to get back into riding a while ago, I thought about at some point going and doing some of those rides. Since I’m not really doing any road racing this year due to being a bit too busy with work, I decided to try to tackle some of these long and hard rides that I’ve wanted to do for a few years to keep me motivated and riding hard. First up is the Assault on Mt Mitchell.  For a bit of background, Mt Mitchell is the highest point east of the Mississippi ending up over 6000 ft.  And about an hour from where I grew up.  So starts out sounding a little intimidating.  The ride itself actually starts in Spartanburg, SC and you then spend the first 75 miles riding along rolling hills until you reach Marion, NC.  From Marion, you go up 5000 ft over the remaining 25 miles.  Okay, lots of climbing when you’re already tired.  This sounds awesome.  I’m in. The route map Elevation gain
Preparation and Pre-Ride I signed up for the ride back when registration opened in March.  From that point, I received a steady stream of emails detailing the training rides that they offered and suggested including things that covered a lot of the route.  Living about 900 miles away, those weren’t an option. So I basically did a pretty typical set of spring riding for me; stretched out some rides a little more to get more rides in the 80+ mile range instead of 50-60s but no real hill work, etc. Given that my parents still live in NC, we decided to make a family trip down to see them.  So I shipped my bike via FedEx to my dad’s office (unnerving!) and we flew down.  We arrived on Saturday, I put my bike together and did a little loop on Sunday to stretch the legs and shake down the bike after reassembling it.  All good.  I packed everything I needed, the bike survived being shipped, and my legs even felt decent with the lack of riding I had done the week before. Of course, up until this point, the weather forecast for the ride on Monday was looking less than great.  Showers and thunderstorms through the day.  Because riding 100 miles in the rain is fun.  Ugh.  Luckily, after riding in some sloppy drizzle on Sunday, the forecast for Monday magically got better.  I’ll take it! The Start Given it’s about an hour and a half from my parents house to Spartanburg and roll out is at 6:30, we stayed at the Marriott around the corner from the start on Sunday night. So Monday morning, I woke up super early and headed to the start with plenty of time.  Breakfast was my first rice cake of the day (the classic egg + bacon recipe for this batch) although in hindsight I should have gone for something more.  As I had picked up my packet and number the night before, I didn’t really have to do anything other than get to the start which was nice.  As I did so, the size of the event really started to become clear, around 1000 cyclists all told. Early morning window reflection Cue sheet on the top tube Cyclists gathering at the starting area
I made my way towards the front of where people were lining up.  We had the entire street (four lanes) and I wasn’t going to get caught up in the back.  I had modest goals for the event — stick with the front as long as I felt comfortable but mostly in it to finish.  Time wasn’t at the front of my mind as I was thinking of it as a ride, not a race really.  As the countdown got to zero, we took off.  And the front went fast… we we[...]



Thoughts on DevOpsDays NYC

Sat, 19 Jan 2013 17:01:52 GMT

I’m currently on the train on my way back from DevOpsDays in Brooklyn. The conference was great — lots of smart people facing a lot of similar problems and trying to see what we could learn from each other. The scale was small, with only like 100-ish people present and not a ton of huge, in your face sponsorship. And the venue was a college campus. And so I kept making these comparisons in my head to LUG meetings, installfests and small scale Linux conferences.

Obviously the subject matter was a bit different — talking about and thinking about running large scale production infrastructures is a little bit different than the next cool Linux distribution. This tended, I think, to more discussion around patterns and best practices than about the specifics of “you should do X to get Y to work”. So a higher level and more abstract discussion.

The composition of the audience and attendees was a pretty similar make-up. Linux events always had a strong majority of the attendees who self-identified as sysadmins and then there tended to be a smaller number of developers. And many of the latter group had ended up in that camp due to necessity. The breakdown for DevOpsDays felt pretty similar with an interesting twist where there were speakers who said they were (paraphrasing) “developers first and fell into operations because they needed to”.

One thing that felt more evolutionary than anything else was that the side channel discussion for the event took place on Twitter rather than on IRC. I have (fond) memories of many conferences where attendees sat in an IRC channel and then basically continued to interact on IRC long after the conference had ended. In fact, I made many friends in this fashion. Similarly there was an ongoing discussion on Twitter using the #devopsdays hash tag and I have followed (and am being followed by) a number of the other attendees and hope to keep in touch and call them friends in the future.

And maybe the thing that struck me the most strongly was where people were “from”. Not in the sense of where they lived but rather where they worked. The attendees were almost all from startups. We were in Brooklyn and not the heart of downtown Manhattan, but NYC is probably home to more financial services companies than anywhere else in the world. And all of those companies have *many* people working in software dev and operations-y roles. But they weren’t there.

So it feels like “the DevOps movement” is going through a similar growth and evangelism pattern as open source and Linux did years ago. Maybe that’s why it feels so comfortable to me.

Originally published at Jeremy's Thoughts. Please leave any comments there.




Can’t code with AWS outages so blogged instead

Mon, 22 Oct 2012 23:07:56 GMT

Although I haven’t really talked about it here, I joined a new startup a couple of months ago called Stackdriver where we’re working on building a hosted solution to make infrastructure monitoring and management suck less for users of the public cloud.  After a having to duct tape the various pieces together a couple of times now, it’s super clear that the need is there so it’s exciting to be working on solving it.  More on the side of being at a very early startup to come in the future.

Today I had planned to do some work around some of our provisioning and deployment code and Amazon had another EBS outage making the AWS API pretty unavailable for much of the afternoon.  So after doing some other things, I took a look at what fails along with EBS to help us remember what fails along with EBS and thought it was interesting enough to share.

Originally published at Jeremy's Thoughts. Please leave any comments there.




A repo for the chef-omnibus packages

Wed, 30 May 2012 00:28:25 GMT

I finally got around to trying the Chef omnibus installer and it’s a step up from what I was doing previously but still not great.  Grabbing a shell script with curl or wget and piping it to your shell is an anti-pattern which I wish had never taken off.  Luckily, in this case, the shell scripts is just pulling down an rpm and installing it.  One step nicer would be if there were just a repo that you could use via yum and have things a yum install chef-full away.  And as I thought that this afternoon, I remembered the baseurl support in createrepo.  Thus, without further ado, I’ve thrown together a quick set of repos that just point to the files in the opscode s3 bucket and minimizes the amount of storage I have to do (image)  If you want to use them, just drop a file into /etc/yum.repos.d named something obvious like chef.repo

[chef]
name=Chef Omnibus Packages
baseurl=http://katzj.fedorapeople.org/chef-omnibus/el$releasever/$basearch
enabled=1
gpgcheck=0
#gpgkey=

I’ve only tested the EL6 x86_64 package but I went ahead and created the repos for EL5 and EL6, both i686 and x64_64.  Yes, the packages aren’t signed right now.  Hopefully that’s something that can be remedied relatively easily.  And even better would be if Opscode would just integrate the simple call to createrepo into their build process for the omnibus installer.

Originally published at Jeremy's Thoughts. You can comment here or there.




A Puppet User Trying Chef

Thu, 19 Apr 2012 15:50:18 GMT

I have a decent amount of experience at this point with puppet both from experience using it to manage the infrastructure running Fedora as well as setting it up at a pretty large scale at HubSpot.  But in a new gig, I decided it was worth rounding myself out a bit and giving chef a try.  Not out of any deep seated dislike of puppet but there are a few pieces that I’ve continued to run up against which are a little grating and so I figured it was worth broadening my horizons.  The nice thing is that both are fairly successful open source communities and realistically, as long as you’re using a system, you probably can’t go that wrong or switch in the future. Side-note: I’ve also been playing with Michael Dehaan’s new project, ansible which is also interesting. But I don’t think it’s mature enough to use for a production environment yet and I also was mostly interested in it as a better remote execution layer as opposed to another full fledged config management tool. But yeah. It’s there.  It’s interesting. I’ll probably write more about it later. With a little bit of chef time under my belt, I have to say that I’m not struck by drastic differences.  The terminologies are different, the DSL used on the config side is a bit different but they act pretty similarly and you can get either of them to do what you want.  That said, there are a few things (good and bad) that I’ve noticed about chef and figured I’d share for others who are looking at deciding for themselves.  Note that a few of the things in the dislikes section may well just be me missing something and being a n00b… suggestions welcome! Things I’ve Liked  Hosted Chef is a very very nice option to have.  Props to the Opscode team for building an infrastructure to run the server side for youand especially for making the barrier to entry nearly zero by letting you manage up to five hosts for free.  Given some of my headaches around running a puppetmaster previously, I’m glad not to be having to pull together everything to run a chef server Knife is actually pretty cool.  I was skeptical before using it but it does a pretty nice job of encapsulating a lot of common tasks for you Knife gets really cool with the addition of the ec2 plugin.  Launch servers, register them with hosted chef and have them ready to go.  I’ve built all of the surrounding bits and as the environment I’m dealing with grows, I think I’ll grow out of being able to use knife ec2 effectively, but it’s great for an easy starting point Chef solo seems to work okay and have a few niceties over a master-less puppet setup but I didn’t spend much time with masterless puppet, so it’s probably just that I didn’t find the related nice pieces Things I’ve Disliked / Been Annoyed By The package support in the Fedora/CentOS/RHEL universe is pretty poor.  I realize that all the cool kids use Ubuntu these days but tons of server infrastructures are not.  Todd does a great job with the puppet (+ ecosystem) packages for Fedora and EPEL. Would love to see someone do similar for all of the Chef stuff A lot of the cookbooks that are out there and published are Ubuntu specific. Even the ones which strive to work across distros often end up coercing the Fedora universe to look more like Debian.  Which isn’t necessarily a path I want to go down Probably just a side effect of this but a lot of cookbooks using things which aren’t the standard init system (eg, depending on runit) knife-ec2 makes you think you can get away with using it but I keep tripping across things it doesn’t support and making me consider abandoning it Trying out cookbooks from others drives me crazy.  I’m pretty sure I’m missing the good workflow here but polluting my checkout by adding vendor b[...]



Monitoring Driven DevOps

Tue, 28 Jun 2011 00:23:45 GMT

Wrote up a nice post that maps pretty well to the Ignite talk I gave at Velocity about using using monitoring to help drive your infrastructure development.

Go check it out over on the HubSpot dev blog

Comments (image)



Velocity 2011

Mon, 20 Jun 2011 01:44:44 GMT

I spent last week out in California for the O’Reilly Velocity Conference.  It was in Santa Clara, which I hadn’t been to and frankly, I would be perfectly happy to not return.  Parts of California are nice, Santa Clara is an office building wasteland.  No good food options, nothing really going on, etc.  But I was there for a conference and not for other stuff, so it sufficed. The conference was actually very good.  It has been a few years since I’ve been to a conference between grad school, my daughter being born, and being at a startup where conferences weren’t the priority.  But it was good to get back to it.  Had a lot of good hallway conversations with people about things that are relevant to us and saw a lot of good presentations.  And Velocity is especially relevant to me at this point as it was all about various web performance and operations stuff.  Where, unsurprisingly, there’s a lot of cool stuff going on. I mostly kept to the more operations-y tracks just because they map better to what I’m currently working on.  I’ve come away with a bunch of things to look into and posted a whole bunch of choice quotes over on Twitter, but a few takeaways boiled down for here would include If you’re using a public cloud provider, plan for things to fail.  Build your systems expecting it and you’ll have less pain. HubSpot is doing an awesome job with post-mortems.  DanM actually posted a great blog post over on our dev blog about things we’ve learned from doing a lot of them. Everyone complains and focuses on javascript performance but that’s misguided.  The bottleneck is the DOM.  Interestingly, none of the browser guys talked about that apparently DevOps has mostly been about putting developers into ops (hi!) but also needs to be about putting ops into dev Web performance has been very successful in tying itself to business metrics.  Weirdly, operations has overall been less successful at that There’s a lot of work going on to help with debugging and working on webapps for mobile platforms.  Very cool. None of those are particularly earth shattering revelations, but still good to see/hear. Also, on Tuesday night I did a talk for the Ignite track.  So 5 minutes, 20 slides, auto-advancing.  My topic was “Just Too Late” and was largely around some things I’ve discovered transitioning into a role where I’m doing more ops stuff and the fact that I feel like I get to things too late.  But then turning it around and showing that’s not really so.  Stay tuned for a longer blog post on the topic.  But the talk went really well.  It was fun, a lot of positive feedback and was good for me to get back to it.  Looking forward to submitting some (full-length) proposals for talks for some conferences later this year. I also had a few thoughts on the way conferences have changed since I last went to one Twitter really is a pretty big game changer.  Lots of conversation on twitter during the conference about which sessions were good, useful tidbits from sessions, etc.  I actually felt that the experience was pretty strongly enhanced by it Conference wireless still sucks.  But you can get decent data now for devices and avoid the use of the conference wireless entirely.  This made it easier to stay on twitter during the conference An iPad (or other tablet) is a pretty perfect device for looking at stuff during a conference.  It sits on your lap so you can just check it sporadically, the battery lasts all day, you can get data from a cellular provider, and it’s reasonably fast. Anyway, good time was had.  Thanks to all the people that I met and chatted up.  And hopefully it won’t be as long before I make it to another conference Comments [...]



My new role

Thu, 03 Feb 2011 16:10:11 GMT

I’m still at HubSpot but my role within the company has changed a bit over the past few months.  Related to the article that Yoav wrote which was posted on onStartups today about how we’re trying to better empower our engineers and teams to really own things, I’ve shifted my focus some.

Instead of working on the product which is front and center to all of our customers or even working on the free tools at grader.com that millions of people use, I’m now instead focused quite a bit on various infrastructure related things for us. Obviously, I’ve done some of that all along, but at this point, it’s my primary job.

It’s a lot of fun. We are heavy users of EC2 and some of the other Amazon services. We also are using Rackspace Cloud some. And I wouldn’t be surprised if we add another provider in the future. So there is a challenge in making all of these environments look the same for the rest of our dev team as well as our on call folks.  We’re also working to make it so that we can easily continue to scale out as our compute needs increase.  All the sorts of things that I’ve spent some time thinking about over the years, but there’s no theoretical here — we’re really deploying, managing and everything else a pretty large distributed system. We are using a fair bit of open source stuff in addition to building some stuff ourselves.  The first thing was obviously ami-creator but there’s more to come almost certainly. In addition, we’ll probably be doing some work and submitting some patches to improve some of the tools and things that we use as it makes sense to do so.

And as we we are growing like crazy, I’m looking to hire some people to join my team to help us get even more things done. If I were writing a job description it would probably include bits and pieces like Linux administration, python, puppet, probably devops (as it’s something that’s in mind), cloud automation (… even though I still hate the word cloud), release and build tooling, monitoring, and more. Sound interesting? Drop me a line and let’s talk.

Comments (image)



2010, The Quick Look Back

Thu, 13 Jan 2011 03:10:18 GMT

I could give some excuse about being busy, but everybody does that.  Let’s do the rundown of big events of 2010

  1. Finished and submitted my thesis to complete the work necessary to finish up the SDM program and graduate in January.
  2. My daughter was born in February.  There are plenty of pictures on Facebook if you know me and are on there (mostly taken by my wife)
  3. I’ve continued working at HubSpot where there’s no shortage of work to be done and it seems like always something big going on or coming up
  4. Picked up cyclocross racing in the fall.  Read all about it. Since apparently I managed to blog about that.

If you wanted more details, well, there was a lot more on twitter.  But hopefully going to get back into the habit of some blogging again for the new year.  Then again, I’ve said that before.

Comments (image)



I’ve moved my bike blogging

Fri, 15 Oct 2010 02:49:50 GMT

I’ve been wanting to play with tumblr, so I’ve set up a new blog for my bike blogging to try it out. Check it out for exciting race reports, some video and probably some other random thoughts on cyclocross as I begin my inaugural season of cyclocross racing.

There might be some other reorganization and moving around here as well in the future when I have a little bit of spare time. Which, since I’m racing cross, might not be for a few months (image)

Comments (image)



Stop Using the Word “Cloud“

Thu, 21 Jan 2010 05:11:00 GMT

The more I see it, the more I want to just completely see the usage of the word “cloud” go away. While it’s somewhat of a cliche to say so, it’s a term that has a very hazy and non-concrete meaning. So whenever you start to use it, you immediately end up in the “well, what is a cloud” discussion. And thus, I have a set of suggestions for those places where you might have wanted to use the word “cloud” to instead use something which actually has meaning.

  • If you’re using cloud to refer to EC2, use EC2 instead. It’s concrete and it means very real things about your deployment and scaling models as well as how you’re managing your infrastructure.
  • If you’re using cloud to refer to some service which runs over the Internet, either refer to the service or just say the Internet. You don’t store your mail “in the cloud”, you host it with Google apps. You don’t backup “to the cloud”, you have your backups stored over the Internet with Mozy or Carbonite.
  • If you’re using cloud to refer to the idea of some hosted application platform, just say the platform. You don’t run your python app “in the cloud”, you run it on AppEngine (or something else).
  • If you’re using cloud to mean that you are using virtualization and have some management stack on top of it, then please just say you’re running in a virtualized environment.
  • If you’re using cloud to refer to having your server infrastructure hosted in a virtualized environment by someone else, again, just say you’re running in a virtualized environment.
  • If you’re using cloud to refer to a “visible mass of little drops of water or frozen crystals suspended in the atmosphere”, then congratulations, you can continue to use the word cloud. And thanks to Wikipedia for the definition

Following this simple idea will let you avoid the otherwise impossible to avoid discussion of the semantics of the word “cloud” and what you happen to mean about it and how you might be wrong and … This then means you’ll be that much closer to achieving whatever goal you hoped to achieve as you’ll spend less time talking and more time doing. And as an added benefit, you’ll avoid getting grumpy emails from me about the fact that you’ve used such a terribly over-used and under-meaninged term.

Comments (image)



Time to get back into a training routine

Sun, 18 Oct 2009 23:02:34 GMT

As a road rider and racer, my cycling season tends to wind down about this time. If I were to start racing cyclocross, I’d extend it out, but for now, I’m staying out of that. The past two years, I’ve marked the end of my season with racing at the Jamestown Classic down in Rhode Island. This year, a combination of the fact that I really kind of needed to work the day and also that my fitness wasn’t really where it should have been for a race led to me skipping it. Now I’m a little bummed that I did, but c’est la vie. I’ve spent most of the past six weeks generally riding just for fun and without any real training goals in mind, although I have been watching the power numbers on my shiny new powertap out of curiosity.

Looking back on the season, it was one that was both successful on some fronts and utterly not on others. I did a good job of keeping up a good base training routine through the winter but then ended up doing little in the way of racing over the course of the spring and summer. First it was waiting for the new bike, then it was being busy, then the weather sucked, then I got hit by a car, then travel, and then the season was over. Even though I didn’t race much, I felt like I was a lot better prepared for the races I did do and that my fitness was higher as a result of the base training through last winter.

So I think it’s now time to start easing myself back into a bit more of a routine in preparation for the winter of base training. I picked up a new trainer to replace the freebie I had been using that’s significantly quieter. Last winter, I was able to do trainer time in the evenings, but with my current schedule that seems unlikely so I’m going to start getting up a little earlier to get time in before the ride into work.

Set it up last night and had the first ride on it was this morning and it’s pretty nice — quieter than the old one and seems a bit smoother as well. I’ve got a pretty good set up to start with to be able to watch DVDs or online video. I’m then streaming the audio to my iPhone with AirPhones so that I don’t have to have a long headphone cable or worry about turning up the speakers really loud. Today was watching some TV via Hulu and then a Spinervals DVD. For the latter, though, I need some better music. What do other people listen to as a good upbeat playlist for time on the trainer or even general race warmup, etc?

Comments (image)



EC2 and Fedora: Still stuck at Fedora 8

Fri, 16 Oct 2009 19:46:24 GMT

Amazon’s EC2 service is great for being able to roll out new servers quickly and easily. It’s also really nice because we don’t ever have to worry about physical hardware and can just spin up more instances as we need them for experimenting or whatever.

Unfortunately, they’re still stuck in the dark ages with the newest AMIs available for Fedora being Fedora 8 based. With Fedora 12 around the corner, that’s two years old — something of an eternity in the pace of distribution development. I’d love to help out and build newer images, but while anyone can publish an AMI and make it public, you can’t publish newer kernel images, which really would be needed to use the newer system.

So, if you’re reading this at Amazon or know of someone I can talk with to try to move this forward, please let me know (katzj AT fedoraproject DOT org). I’d really strongly prefer to continue with Fedora and RHEL based images for our systems as opposed to starting to spin up Ubuntu images for the obvious reasons of familiarity.

Comments (image)



Why do all deployment systems suck?

Wed, 07 Oct 2009 02:50:18 GMT

At HubSpot, we have a pretty wide array of different things being used for the webapps running behind the scenes. This isn’t surprising. There’a also some home-grown scripts (in python, as that’s the scripting language of choice… something I’m not complaining about) to take care of deploying the various webapps. It works, but I really want to get it doing a bit more so that it’s more useful and also get the different scripts doing a bit more sharing of code so that we can improve one place and get the benefits for everything. Given that this seemed like a pretty typical problem, I figured I’d take a look and see what open source projects exist out there to see if any of them were suitable or could be at least close to a good fit for what we need and want. Unfortunately, I was kind of disappointed… Capistrano seems to be the big player in this arena. It was originally written for Rails and still very very strongly shows that heritage. This isn’t necessarily bad, but it makes it a lot harder to get to work if you’re not doing something that’s rails-like. There are some people who have gotten some things working with Java app deployments for tomcat, but they all feel a bit hacky. The other downside for me/us is that Capistrano is very much Ruby-based, both in how its own deployment language looks as well as some of the “how it depends on things working” aspects. Also, the fact that it’s written in Ruby and thus a little bit more difficult for us to hack on if/when we run into problems is a point against. So it’s probably a non-starter for now, or at least a pretty difficult sell Fabric is written in python and seems to be following in the footsteps of Capistrano. Right now, it’s far far simpler. This is in some ways good but some of the pieces that we’d want (eg, scm integration) aren’t there and so I’d have to write them. And I’m not sure if the Fabric devs are really interested in expanding in that way; haven’t sent email yet, but planning to tomorrow to feel it out. Config Management + Binary deployment is the approach taken in Fedora Infrastructure for app deployment and it seems to be working pretty well there. It might be something to get to eventually, but that’s going to be a longer term thing and I’m not actually convinced that it’s really the best approach. For Fedora it grew out of only a couple of things which could be considered “webapps” and a lot of system config that has turned much later into more webapps. It also pre-supposes a bit more homogenous of an environment than we use at HubSpot from the work I did there Func is something that a few people have been working on that I keep wanting to find a use for but it seems a little less well suited to doing a lot of java app building/deployment given that it’s more https/xml-rpc based than shell based. Roll your own is what we’re doing now and what it seems like is pretty common. I don’t necessarily like this, but it’s certainly the path of least resistance So, what am I missing? Is there some great tool out there that I haven’t come found that you’re using for Java (and more) webapp deployments? Bonus points if its python-based and pretty extensible. Comments [...]



10 Year NCSSM Reunion: Has it really been that long?

Tue, 06 Oct 2009 02:19:42 GMT

I spent the weekend at my ten year high school reunion. A lot of people that I’ve mentioned this to have been somewhat surprised or responded along the lines of “I wouldn’t go to my high school reunion”. Which I can understand. But Science and Math is different than your typical high school. As a two-year residential magnet school, you end up getting to know people pretty well — it’s basically like going to college a couple of years early, only with a little bit more supervision. And so it’s the same sort of bond and friendships that a lot of people end up getting in college. Kara and I flew down on Friday morning and then met up with a friend for lunch. We then spent some time just kind of relaxing before meeting up with a group of my friends from high school for dinner. It’s a good group although we don’t get together anywere near as frequently as we used to… for the first few years after high school, we made it a point to get together around New Year’s or at some point over the holidays. This has fallen off as we’ve all moved increasingly far apart distance-wise and have increasingly busy lives. It was good to get together and catch up, though, and especially in a smaller group environment. Later in the evening was the sort of opening party/reception type thing and my class had a surprisingly good turn-out. The event was held at a bar in Durham (Tyler’s) which was fine. It was a little weird seeing people smoking indoors as that’s something kind of foreign after living in Massachusetts for almost six years. This probably contributed to the scratchy throat feeling I had the rest of the weekend. As I walked around, I was able to see and catch up with a ton more people; both that I had been friends with through high school or people I barely talked with. It was really nice as pretty much everyone is doing something that is kind of interesting now and it was cool to talk with them. None of the usual crap that people complain about with high school. Saturday was the main day of reunion stuff but my group of friends started earlier with breakfast at Elmo’s on 9th St. As usual, Elmo’s was good and we got there just in time to beat the majority of the rush. Then, it was down to campus and we watched a presentation about some plans to expand the school. I’m not really sure how I feel about it. On the one hand, it looks like a fair bit of work is going into making sure that things are nice and really trying to turn what had been a hospital hastily converted to a residential high school into a very nice residential high school. On the other, more people in the same sized space could be a bad thing. A day later and I’m still not sure where my opinion falls on the expansion. The rest of the day was filled with catching up with more people, walking around campus to see what had changed (both a lot and very little at the same time), our slide show (always fun to watch again) and the like. Kara and I then skipped the evening’s festivities so that we could grab dinner with my parents as we haven’t seen them in a while and it’s unclear exactly when we’re going to be down again. It was a low-key evening and then rather than attend the late night party, we just watched Valkyrie in the hotel room. This morning we slept in a bit more and then headed to campus for what had been billed as the faculty brunch. The location stated for it had been a little bit odd, and when we got there, we learned why — it was actually in PFM (the cafeteria). Also, the nu[...]



Commuting Pedal Failures

Tue, 15 Sep 2009 02:34:31 GMT

One thing that’s quite nice about the new gig is that the office is in Kendall Square. Much, much, much better location-wise than Westford. It means that my commute is just about seven miles which is quite nice to do via bike. Also, if the weather’s bad or I feel lazy, I can take the bus to Alewife from right outside my house and then take the train in.

Unfortunately, I’ve now had two weird pedal failures in the past week. Last Thursday, I was leaving the office and clipped in. As I got about a block away, I noticed my foot moving weirdly on the pedal. As I pulled over to check it out, it became clear that the cleat was stuck in the pedal. After some investigation later, I realized that I lost one of the two screws holding the cleat into the plate in the shoe. It looks like the plate where the screw went in is actually pretty stripped. And in getting the cleat and shoe disengaged from the pedal, I essentially had to take the pedal apart so I decided to switch the pedals out for the plain SPDs instead of the slightly fancier SPDs that were on there.

Today, I was riding home and realized about halfway home that one of the pedals was coming unscrewed from the crank. I made it home without incident and re-installed the pedal without any noticeable problem, but I’m going to be keeping an eye on it over the next few days. Hopefully the crank isn’t stripped — it looked okay, but at this point, I’m a little cautious of it.

Maybe I should look at building a new commuter bike sooner rather than later (image) Although I really would like to get the Redline to last another year to year and a half.

Comments (image)



Review: Amazon Kindle 2

Mon, 14 Sep 2009 02:40:21 GMT

I had asked via twitter a while ago for opinions on the Kindle as I had been thinking off and on about getting one for a while. Responses were very positive with really one exception: the DRM aspect. And it’s something that bothered me, but I decided that the convenience benefit was enough to outweigh that for me at this point. Especially when I considered that I often buy books, read them, put them on a bookshelf for six months to a year and then drop them off at a book drop. Re-reading isn’t my usual, instead buying new books is Especially as the primary purpose for the Kindle is more for pleasure reading as opposed to more technical stuff Since I knew I’d end up reading a lot while I was on vacation, I figured I’d take advantage of Amazon’s good return policies and try it out. If I liked it, great. If I didn’t, I could certainly hit a bookstore to get “regular” books and return the Kindle when I got back home. As lots of people have said, the packaging of the device is very good — Apple-like is really the best way I have to describe it. They really want you to buy into a full experience. I turned it on and had already set up a few “free sample” books to be on the device. And reading on it has turned out to be a great experience. The page turning was a little bit distracting at first with a little bit of a flicker, but after half an hour or so of reading on it, I don’t notice it any more than I notice turning the page of a regular book. One thing that I didn’t find at first (because I didn’t read the documentation) is that you can change the font size — this was pretty handy as I can definitely handle smaller than the default — I’m currently using the smallest and am pretty happy with it. One of the obvious questions with the Kindle is battery life. With the wireless off, it was incredible (I didn’t apparently need to charge during most of my 10 day trip with it sporadically on). Leaving the wireless on I need to remember to charge it every three or four days. But the charge is pretty quick. Unfortunately, it uses microUSB instead of miniUSB, so it’s another cable to have to be carried bringing me to three (miniUSB, microUSB, iPhone). The selection of books seems to be pretty good. Not everything that I’ve looked for, but the vast majority. Pricing is okay, usually a little cheaper than the price for the dead-tree version and the over-the-air delivery is wonderful. Especially when sitting in an airport between flights. In terms of reading on it, the screen is incredibly nice. I had my doubts, but they’re entirely gone. Reading in bed? Check (just be sure there’s a light). Reading outside beside the pool? Check. On a plane? On the couch? … And so on. The screen is incredibly readable and I don’t end up with any of the eye strain problems that I sometimes get after staring at a laptop screen or a monitor for long periods of time. How well it worked in sunlight is one thing that really surprised me. Obviously it’s supposed to be one of the strengths of the screen, but I’m wary of marketing-speak But I really had no problem sitting and reading it even in direct sunlight. Downsides? There are a couple. The biggest is the DRM stuff. It’d be nice to be able to share books (eg, if Kara decided she wanted to get a Kindle also so that we could both read the same books). It’d also be nice to have some sort of library functionality, althoug[...]



Garmin 705 Tip: Sending Routes Between Devices

Mon, 07 Sep 2009 18:51:56 GMT

While I think I’m pretty good with the capabilities of the Garmin Edge 705, I learned a new trick with it last weekend. We were planning to do a century up to Cape Ann and Gloucester and I had mapped out a route with Bikely and loaded it onto my GPS. I sent out the link to everyone else who was planning to come, but Charles had not pre-loaded it onto his. Instead, he said that you can actually transfer routes, courses, etc between devices wirelessly!

It’s actually pretty simple — you hit the menu button, go to Settings, then “ANT+” and then Send on the device doing the sending and Receive on the one doing the receiving. And then, voila, you’ve sent the route between Garmins.

Very very cool, and something I’ll definitely have to keep in mind for future rides to new places as more and more of the team start getting the 605 and 705.

Comments (image)



First Week at HubSpot

Fri, 04 Sep 2009 23:35:28 GMT

As I wrap up my first week at HubSpot, I have a few observations that are at least sort of interesting.

  • Real hardware. I’m pretty happy with my current laptop so I just got a desktop machine to use at work. The box I got is a Dell quad core with 8 GB of RAM. Nice box overall and Fedora installed with no problems. The nVidia graphics work fine for 2d and even xrandr seems to be doing the right thing. One thing that is annoying is that Dell is still shipping machines with VT turned off in the BIOS. Once I turned that on, though, KVM is also working pretty well on the box
  • Windows is both just as annoying as ever, less annoying and more annoying. You can run it in a virtual machine without real problems. But installing things, the terminals, etc are all still a pain. Stability is a bit improved. The whole “run as administrator” nonsense is a real pain when you’re trying to get a lot of stuff going.
  • Coming in at the end of the scrum cycle seems to sort of be a good thing. Get to see the final push and then the demos from that cycle followed by getting to sit in on the planning for the next sprint. I won’t be on a scrum team until the next sprint and so hopefully I’ll have a better frame of reference¡
  • Commuting to Kendall Square works really well for me. Okay, I knew this from riding into MIT but it’s still a takeaway. The bike ride in is a nice length; shorter would be fine, but longer really isn’t as practical.
  • Complex build processes exist everywhere and are despised everywhere. But it always seems like a build and deployment process is the last thing cared about.
  • I’m having a lot of fun being back in a startup environment.

So yeah, all in all, its been a good week. Now for a long weekend. Two four day weeks in a row for me I guess.

Comments (image)



The New Chapter Begins

Wed, 02 Sep 2009 02:26:36 GMT

The new chapter begins… today was my first day working for HubSpot.

It’s a big change for me as I’ve been doing pretty much purely (fairly) low-level operating system work for a decade now. Going to a company that’s doing much more web development is making me shift how I think about everything from considering using Eclipse rather than a combination of Emacs/vim/terminals to the languages I’m writing in and the types of code I’ll be writing. And I think it’s a change that I need — I’ve been feeling a bit stagnant and so getting out of my comfort zone should help a lot.

Also, I think that HubSpot is doing some interesting stuff and I’m glad to be joining the team to help out in a variety of different ways.

Comments (image)



Beginning A New Chapter

Thu, 27 Aug 2009 15:41:37 GMT

The end of one chapter and the beginning of a new one for me. Today is my last day as an employee of Red Hat. I still remember walking in the door for my first day at Red Hat and having Nalin set up my account so I could get started as Preston was a little bit late getting in that morning. It’s been a great eight+ years across five offices and two states working with lots of great people.

During that time, I’ve also had the opportunity to play a big role in the development and growth of Fedora. While the start was somewhat rocky, I think we’ve now built up an incredibly strong community that successfully releases a whole distribution (arguably, several!) on a regular schedule. And within that community, we’ve grown a pretty awesome set of leaders to continue to drive Fedora forward.

While I’m planning to still keep at least in touch with the goings-on of Fedora as well as running Fedora in places, I certainly won’t have the time to spend on it that I do today. I hope to keep in touch and see people at conferences and events from time to time. But right now, I’m looking forward to what’s next for me. And for those wondering, it’s something pretty different really. More on it next week..

Comments (image)



2009 Seacoast Safari Recap

Thu, 20 Aug 2009 02:46:46 GMT

Better late than never right? Here’s a quick recap from this year’s Seacoast Safari that I had started and has been sitting in the drafts folder… For the third year, I signed up for the Seacoast Safari ride to raise money for research and treatment of Cystic Fibrosis. As I’ve previously mentioned, this ride has a bit of a personal connection for Team Quad as one of the guys that rides with us, Chris Kvam, actually has CF. Not that you’d know it from watching how strongly he rides. This year, Quad had an even larger team than last year with 26 riders. For me, the thing that made it even more fun is that I had convinced my dad to fly up and do the ride with me. Originally, the plan was to have him ride my Merlin and I would ride my Cannondale. The previous week’s accident made that not an option. But we found another team bike for my dad to ride and all was well. Also, Kara served as the official photographer for this year’s event. You can see all of her pictures on flickr Saturday started off a little bit drizzly, but it looked like the majority of the rain had passed us. The roads were still a little wet, though. I had decided to take it easy and to mostly ride along with my dad which worked out well for riding with a number of the other Quaddies. All was going well with about 10 of us riding together until Jon hit a rock and was thrown off-balance and went sliding across the pavement. Ugh. Not the way to start a charity ride. But we made sure that the paramedics were taking care of him and we continued on. The next mishap, just a mile or two later, was a flat that for some reason was quite difficult to successfully change. Finally it was fixed and the rest of the day went by pretty uneventfully and the sun finally came out. As we got to UNE in Biddeford, we saw Jon standing there after having been discharged by the ER and brought up by one of the Cystic Fibrosis Foundation staff. We had the barbeque lunch and then watched some of the coverage of the Tour stage from the day. Dinner was the typical pasta dinner that has characterized the Seacoast Safari along with an award ceremony and silent auction. A nice impromptu addition to this year’s event was a few people auctioning off their prizes to raise even more money. Very cool! And Team Quad was recognized for the fact that we both had the largest team and had the largest fundraising amount of any team. Overall, the amount raised this year was over $100,000 making it significantly more than previous years. We headed back to the dorms where we mostly sat around and talked bike geekery and then headed to bed to wake up early the next morning again. Sunday morning came along and was quite sunny and perhaps perfect from a weather standpoint. We had breakfast and were then on our way. Again, I was content to take it easy and ride with my dad for the majority of the time. A number of other people were content to keep the same pace and so we had a nice group of about 8 Quaddies riding along at a fairly relaxed pace. We took in the scenery, we sang and enjoyed a wonderful day on the bike. At one point, I went to the front of the group and basically sat there for about 12 miles setting a steady pace into the wind. It was a great time and a good way to get a good tempo workout without having to really push it. We made it back to Newburyport and wrapped things up. Stood around talking with[...]



Maui Trip: All the Rest

Thu, 20 Aug 2009 02:29:43 GMT

After Friday’s trip up the volcano, the focus of our trip shifted to Kara’s sister’s wedding. Kara and I were both in the wedding, so there was a rehearsal on Saturday afternoon. While we were staying in Kapalua on the northwestern end of the island, Lindsey was staying at the Grand Waileia resort on the southern tip of of the island and that was where the wedding was going to take place. So about an hour’s drive. So Saturday we grabbed lunch with Kara’s parents and Kara’s other sister (Courtney) plus her husband and five month old again at the Cool Cat Cafe. This time I had a burger and it was a pretty good burger — I see why they are highly rated. We then headed to Waileia for the rehearsal. The rehearsal was quick and basically there was little information really to be shared. Then, Saturday evening there was a rehearsal dinner / welcome dinner (some of Kara’s family that wasn’t in the wedding came out as well and they came to this dinner as well) at the Hard Rock Cafe in Lahaina. And the dinner really was what you expect at a Hard Rock Cafe. That’s not to say it was bad, but the food was predictable. We did have a nice outdoor table to sit at with a good view of the setting sun over the ocean. After dinner, Kara really wanted some shave ice again so we went back to Ulalani’s Shave Ice. We shared a large this time and it was very sufficient for the two of us and again very good. Then, it was late so we headed back and went to sleep. Sunday was the wedding and we were up fairly early. I drove Kara and Courtney to Waileia for their hair, etc appointments. Kara and I grabbed lunch at Longhi’s and I had a pretty good pasta dish — the Fettucini Lombardi. It was a fetticuini alfredo with the nice addition of proscutto and English peas. Very tasty and by going for lunch, the prices were a lot more reasonable than their dinner prices. Then I dropped Kara off for her hair appointment and just read for a while. Then it was time for the wedding. This was actually kind of cool. Lindsey had decided to have various bits of native Hawaiian weddings in the ceremony including a precessional where she was carried in on a chair and accompanied by hula dancers. Very very cool to watch. The rest of the ceremony was nice as well, but it’s hard to beat being carried in on a chair After the ceremony, there was a dinner/reception at the Bistro Molokini in the Grand Waileia. Dinner was good, but the service was very very very slow. While they started off getting things fast, they slowed down quite a bit and it was probably about two hours before our food came out. Oh well, we were in no rush. We eventually headed back (late) to the condo and crashed hard into bed. Of course, after I had driven the hour back, I was wide awake and so I read for a while longer and eventually fell asleep. Monday came and it was raining a little so I decided against going for a ride. But after an hour or so, it cleared up and so we went out to hang out by the pool for a while. I read and swam a little bit. Then, the plan for the evening was to go to the Old Lahaina Luau with everyone. Rain threatened, but held off for the entirety of the luau. It was pretty well done. They did a good job of not just having a mob scene at the buffet for food by sending up a table at a time. The performance afterwards was ve[...]