Marketing your podcast, thoughts

This is a repost of a comment that I left on the /r/podcasts Reddit about how to spend $500 to market a podcast. My response assumed that the primary source of revenue for the show is advertising. The ROI does not work out if that is your only source of revenue, though if you are using your podcast as lead generation for more expensive services, the customer acquisition costs might be worth it.


I’m assuming that you will continue to focus on improving the quality of your show, production and guests, and that this money is for marketing, advertising and promotion purposes only.

I’m also assuming that you are making more than $500/month in advertising revenue. At a $25 CPM you are earning $0.025 per listener download. Each listener would need to download 100 episodes before you’d make back a $2.50 listener acquisition ad spend.

Given that you might be willing to spend $500 each month on advertising I’d guess that you have somewhere in the range of 15,000-20,000 monthly downloads. Encouraging your existing listener base to promote your show might be the most effective method to grow the audience. Find a way to identify your most dedicated and passionate listeners and provide them with shareable marketing materials. Help those dedicated listeners build a street team that promotes your show to others.

I could provide more specific advice if I knew more about your motivations for creating the show, but some broad things to consider:

  1. Who is the potential audience for your show? Really nail this down to a small set of archetypes. “Whitetail deer hunters who are looking for advice.” or “Librarians who are looking for book reviews.” The more specific, the easier it will be to target a potential new listener with ad copy that addresses their needs.
  2. Does that audience listen to podcasts? If you are attracting a new audience, will you need to explain what a podcast even is? Advertising or cross-promotion on other podcasts means that you don’t need to explain what a podcast is, but then you are competing with other podcasts for that potential listener. Facebook and Google Ads are going to reach an audience that might not be familiar with podcasting, asking them to subscribe is a pretty high bar.
  3. Partner with adjacent media properties/companies. Are there established companies that are already doing something similar, but have yet to develop a podcast? Develop a relationship with those companies, pitch them on why a partnership and cross-promotion might amplify both brands.
  4. Are there blogs that are relevant to your topics and show? Find bloggers that might be interested in the hosts or topics. Give them advance notice of the show, write mini-press releases for them so that they have less to write on their own.


  1. Invest in the quality of your show. No brainer.
  2. Invest in visual branding Good design signals quality. Quality develops trust. Advertising to a new audience means that you have to establish enough trust that a potential listener will be willing to take a chance on your show. The goal is to get them to listen to your show, guide them into that pathway with strong visual elements and copy. Do you have a strong wordmark and logo? Are there high quality photographs of hosts, guests?
  3. Create video teasers/trailers/commercials. This isn’t the same thing as just posting the audio for your show on YouTube. Create a commercial, a 30 second clip that highlights why someone should visit your website. Use YouTube annotations to drive people to your website: so that you can encourage them to listen to your program.
  4. Ensure that you have your own website and domain. Telling someone to visit is much easier than telling them to search for your show and subscribe on iTunes, Overcast and the dozens of other apps. Once you have them there you have much more control over how you communicate with them. When I worked at PRX I spoke with the Radiotopia producers about how to convert website visitors into listeners: The slides were well received.
  5. Create landing pages that are tailored to new audiences and are specific to the ad campaigns. I’ve used Unbounce as the landing page location for my Google Ads.

Suggested approach for ad copy For any ads that you decide to run, have the call to action encourage people to visit your website. Do non’t tell them to subscribe to your podcast. Subscribing to podcasts is still a pain and it’s confusing.

Turn and face the strange

It’s time for a ch-ch-ch-ch-change. After an amazing 10 years at PRX I’m leaving my job as Chief Product Officer to rest up a bit, reflect for a while and explore other opportunities. My last day will be November 6th.

Follow your interests

I really enjoy taking photographs, many people know that, but what might not be as known is that I was a heavy and early Flickr user back in 2004–2006. I was an active and engaged participant in the community before Yahoo bought it and things took a different turn and before Instagram was even dreaming of its first filter. I loved Flickr and the community that it helped foster.

In January of 2006 I posted a photo of my six-month-old Ella’s tummy. She ended up with a nasty skin rash after wearing an American Apparel outfit. The support from the community and increased visibility for the issue changed how the company interacted with me. That event helped me understand how power shifts when people are able to speak directly with each other and can reach a large audience without intermediaries.

While I was taking thousands of photos I was hanging out creating software at The MITRE Corporation, a suite of multiple Federally Funded Research and Development Centers or FFRDCs (what a mouthful). A former co-worker and good friend Nathan kept telling me about this awesome organization where he had just started called PRX and how it might align with my interests.

Embrace change

I met with Jake and Steve to learn about what PRX was doing and it stoked my imagination. I remember long conversations with my wife Kim, describing PRX, the potential, the opportunities and the risk. We had a six-month-old, and Kim was at home taking care of Ella, me the in the role of breadwinner, working at MITRE, with a reliable, well-paying job. The kind of job that people rarely left.

For Audrey

The benefits at MITRE were amazing, the work could be interesting and challenging, and many people felt connected to it. The quarterly retirement statements displaying how many times over a millionaire I might be certainly made leaving tough. It was hard looking at a chart with eight figures on it and walking away to the unknown. My father called them the Chains of Gold, and breaking them was hard.

Leaving a 5,000 person company to take a pay-cut at a small, newly formed, mission driven non-profit that operated like a start-up to support public radio producers, while being solely responsible for the support of my family seemed like a crazy change to some of my co-workers. It probably was.

Millions of retirement dollars or meet Ira Glass?

Try, fail, learn. Repeat.

It could have gone all wrong, and early on there were times when it felt like it had, that maybe I had made a poor decision, one that was irreparable. I experienced crashes, both of the application server, disk and data center kind but also those that were very personal and emotional. I came to realize and understand that failure can be an opportunity for learning and growth, but that lesson doesn’t come without bruises, bumps and some scars.

The PRX Marketplace, Public Radio Talent Quest, PRX REMIX on XM,Matter, Dovetail, PubCatcher, Radiotopia, apps for This American Life,Radiolab, taking on the public radio satellite system with SubAuto, dozens of station apps, the Public Radio Player and oddities like Flu Portal andEconomy Story. Coming up with a complete list would be exhausting, but they were all opportunities to try something, learn and improve.

Jake, Kerri, John and the board have been creative and resourceful, finding funders who are interested in supporting the organization and that have allowed us to explore a large number of opportunities and research projects. Thank you for giving us the space to experiment and try.

What did this cat learn?


There are probably dozens of other important or interesting projects and people that I am forgetting to mention. What is most important to me is that PRX has been an organization filled with people who care. People who care about the mission, people who care about their fellow employees and people who care about the success of the producers.

Working here also showed me that politicking isn’t required, that you can care for the people you work with, and consider the personal impact of professional decisions, and still create something successful.

I learned how valuable it is to understand how people might be feeling.

Hugs are important, for those that enjoy them.

Value your time

I found it weird that turning 40 this year mattered. I didn’t think it would. There was something about that round number and marker of passing time that caused me to look to the past and to the future. This past year was filled with moments that gave me pause and opportunities to think about how I wanted to spend my remaining time.

Recently I was organizing photos at home and came across two that caused me to sit back and think. One is of my daughter Ella, from just a week or two after I joined this crazy talented crew at PRX and the other is one that I took of her only a few weeks ago . The difference and change between those two photos amazed me and marked the passing of time in a way that was a stark reminder of just how much had.

My version of the before and after presidential photos. My oldest daughter the week after I started at PRX and one from just a few weeks ago.

I’ve spent almost 25% of my adult life working at PRX and 50% of my career. When I think about all the change that has happened here, it’s overwhelming. It can be hard to see that change though when you are in it, so when I took time to take a giant step back and recognize just how much really has, it was amazing. Those two photos helped solidify it for me, it’s time to try something new.

Beyond broadcast

I’m so excited about this moment for audio as a medium to reach and connect people. It’s crazy to think that my first conference at PRX in May of 2006 was Beyond Broadcast where innovators and industry experts were imagining how to shift the power from entities and organizations to the people.

Inexpensive production tools, podcast distribution, mobile devices, and first class HTML audio support mean producers with initiative are able to reach a direct audience, without gatekeepers deciding what we should hear. Spin up a WordPress or SquareSpace website, post to PRX, SoundCloud or YouTube and bam, their voice can be heard. Doing it well, and creating something that people want to hear, that will always be hard.

A fresh crop of people without a radio background are starting to imagine and create audio. Writers, actors and creative people from many disciplines are starting to understand that audio, and the human voice is a powerful way to reach and connect with people. Distribution and an audience are now accessible to them in a way they haven’t been. It’ll be so neat to see what they create.

Podcasts are just the start.

What is next?

Honestly I really don’t know what my future holds, and that is both scary and exhilarating at the same time. Once again I have found myself in a moment and position where I’m doing something less typical, unexpected and a bit risky. My hope is to spend an appropriate amount of time to reflect on my 10 years working at PRX in digital media, and my nearly 20 years creating and helping design software products before making any decisions. I’m hopeful that things will work out this time too.

Tom Flynn Is The Hottest ‘Jeopardy’ Contestant Ever

The internet is amazing and so is PRX. Both have provided me with opportunities to meet and work with a splendid array of talented individuals. People that I never expected to meet. Working at PRX opened my eyes. I have been dreaming of a period where I could ramp down and unplug, work with my hands, focus on the physical world and my family and hit the big pause button on life for a moment.

So, my immediate plans are to spend time with my family, build some ambitious blanket forts, read books with my kids and do some writing. I’ll also continue training to get my sub 00:20:00 5K time (come cheer me on!) and see how hard I can push my physical fitness. After that, it’ll be time to explore what I might want to work on next. I hear PRX has some pretty neat things bubbling in the laboratory too… Drop Jake a line if that piques your interest. 😉

Data and metrics

I spend a crazy amount of time looking at Google Analytics and spreadsheets, tracking numbers related to PRX. How many licenses, published pieces, producers publishing, listening time, duration, and so on. So I thought I’d wrap up by sharing some of the most important metrics about my time here.

  • 155,000 pieces published (actual)
  • 1,460 chocolate chip cookies eaten (estimated)
  • 8,400 metaphors used (estimated)
  • 241 times Rebecca Black’s Friday was played in the office (estimated)
  • 180,000,000 toe taps (estimated)
  • 550 whiskey sours consumed at holiday parties (estimated)
  • 243 times that the website was down, when it wasn’t (estimated)
  • 323 custom meme photos created and shared (estimated)
  • 3 Filipe’s Burrito locations (actual)
  • 9 children born (estimated)
  • $12 lost playing SpotIt (current estimate)
  • 520 amusing photoshop graphics created for newsletters (estimated)
  • 54,000 happy tail wags (estimated)
  • 2,300 wet dog kisses (actual)

When I see you again…

Jake says you never really leave PRX, so how about instead of a cold goodbye, we make it more of a warm see you later. If we’ve worked together I’d love if we could keep in touch.

I’m always interested in meeting people who want to talk about media, broadcast, disintermediation, open government, municipal government, city planning, Internet and digital distribution. If any or all of that interests you, please say hello. If you also happen to be into bodyweight fitness, I may have found my doppelgänger. You can follow me on Twitter, email me at or find my code hacking on GitHub.

There ain’t no such thing as a free (open government data) lunch

There's no such thing as a free lunch.
Credit: Dana Fradon

There ain’t no such thing as a free lunch — I like that saying. It’s important to remember that it costs money to provide access to information, even public government or municipal information. Someone must pay. But who pays and how much? I like Philip Ashlock’s Tweets so I follow him on Twitter, he also has a nifty proposal called DemocracyMap in the Knight News Challenge. Our NearbyFYI proposal is also in the semi-final round with Philip’s, so when I saw his recent Tweet about “Paying for Public Data” it piqued my interest.

“There are certain elements of our democratic system of government that are so essential to its freedoms and principles that we have to make them as accessible as possible and provide them free of charge.”

That line got my attention and guaranteed that I’d read his entire article. I realize the piece is mostly a response to David Eave’s Tech President post about grant funded projects potentially destabilizing for-profit organizations. I’m not going to weigh in on that discussion as I don’t have strong opinions, but I do want to unpack Philip’s statement that some government information should be free of charge. It’s something that I’ve written about before but I think it’s worth teasing out more.

The government data ecosystem

I’ve spent a lot of time thinking, reading and doing in the Open Government space and I think the following four statements pretty accurately represent the current government data ecosystem:

  1. Citizens must have access to information about their municipality or government.
  2. Access to information has a cost.
  3. Corporations derive revenue from government data.
  4. Citizens pay corporations for services built with government data.

I’m going to focus on statement #2, I have plenty of thoughts about #1, #3 and #4 for a later time, let’s focus on #2 first.

Access to government information has a cost

There are real costs when we interact with our government and request access to information about it. In the non-digital world we need government employees to pull documents from filing cabinets, answer phone calls and handle in-person information requests. Municipal employees attend public meetings, give presentations and summarize reports for us. The human cost to access analog government information is real.

The Open Government movement usually focuses on the digital world, but digital access to government information has real costs too. Proprietary software solutions require licensing fees and support contracts, and open source approach requires knowledgable IT staff, dev-ops and probably developers. There are support costs for open source too, people to help answer questions. There are costs to digitize documents, setup new data publishing workflows and smaller fees for servers (cloud or not), bandwidth, and electricity.

Who is paying the costs now?

I think we often miss an important point when we talk about free, public or open government data — that when we require our government to provide better access to information that it creates a new expense line item. This means that we as citizens pay for improved access — but are we getting a good return on that investment and expense? Larger cities currently pay vendors to provide Open Data portals and information publishing solutions for them. There are open source alternatives like CKAN too. These solutions can work in larger metropolitan areas where the benefits of sharing open data can be realized downstream, but smaller communities will have a harder time justifying the expense. What new open data derived services in a town of 3,000 people will be created that the community values enough to offset the expense?

I live in the Boston area and when the MBTA opened up transit data my commute improved as Google and others stepped in, using tax payer funded data to provide us with useful services based on that data. My experience with the MBTA improved as a result of that investment from my tax dollars. I think my tax dollars are being used wisely. I live in Watertown, Massachusetts (pop. 31,915) and if the town decided to use a vendor like Socrata to provide open data, a very small portion of my property tax dollars would be used to pay for that service. I’ve spent a long time thinking about what services would be created from an Open Data portal in a community of this size and I haven’t been able to think of one that would be worth our tax dollars. When budgets are being trimmed, and it comes down to more teachers or more open data portals, the teachers are going to win and probably should.

My hometown of Millinocket, Maine (pop. 4,506) and Watertown, Massachusetts are probably more like your town than Boston is. Millinocket doesn’t have a public transit system or other large scale services, so it’s harder to explain how Open Government, specifically “Open Data” will help them. When 80% of our towns have fewer than 10,000 people living in them, we should be looking at different models and incentives to get improved municipal information access. Without economic incentives, meaning cost savings — not a new expense, smaller communities will be hard pressed to adopt Open Data tools.

There is another information cost that isn’t often highlighted, I am certainly paying a portion of  a City Clerk’s time when an RFP bid monitoring company from out of state calls my town office and takes 5 minutes to ask for information about new bids. A company that does not pay property taxes in my town. When a fellow citizen emails the Town Manager to ask a budget question it takes time for them to find the information and provide a response. I understand that these examples result in a very tiny portion of my tax bill, but I think it is important to highlight that we’re already paying for data access, it’s just poor quality and inefficient.

So, wait who should pay then?

The Internet is littered with services and software vendors that cost tax payers money, where local government is the customer and our tax dollars help their businesses profit. Code for America references a GovWin report that states $60B will be spent on IT by local and state government in 2013. There is a long list of companies suckling from that wealthy government IT teat. From simple website vendors like CivicPlus, GovOffice and Virtual Town Hall to older established companies like Tyler Technologies, IBM and Microsoft. There are thousands of small mom and pop vendors, mid-tier $5M-$20M companies and fortune 500s that provide solutions for building permitting, parking tickets, accounting and payroll. All of these solutions cost us as tax payers money.

So, you may be wondering how do we get access to municipal information if we’re not paying these companies to provide tools and services? Good question.

Companies like, and eRepublic broker access to municipal information to other companies. They recognized value in the data that our cities and towns generate and spent time and money building up methods to collect this difficult to access information. The Adam and Eve of this approach are Westlaw and Lexis Nexis. I know that there are serious, important discussions taking place about these types of companies and their practice of paywalling access to important documents. To be clear, I believe citizens MUST have access to these documents and information without having to pay a second time. What is important though is that these companies figured out that there are other entities that derive business value from having access to this type of information. What Westlaw, LexisNexis and until only recently JSTOR, couldn’t figure out is how to broker access to data for those that derive business value, while providing access to the public.

To re-appropriate William Gibson’s famous quote “Access to Government data is already here — it’s just not very evenly distributed.”

What are we proposing that is different then?

The approach that we promoting at NearbyFYI blends both of these realities, that there are companies that derive value from better government data and that citizens must have access to it. Access to information has a cost, we just want to shift it over to those that benefit financially in a more efficient way. We see a better way to get high quality data into the hands of those companies willing to pay for it by providing better tools and services for our government. We don’t see expensive “Open Data Portals” as being that solution. We’re going to treat the government as a user, not a customer, helping government employees do their jobs more efficiently. We see a wealth of valuable “dark data” in our municipalities, data that other companies will pay to get access to, we just need to give municipalities better tools to provide it.



Local government is sexy


Local government is sexy.

It isn’t really, but it could be made much simpler to understand. Most of the work that takes place in local government is done in Select Boards, Town Councils and Sub-Committees like Finance and Public Works. Most people could care less what goes on these meetings until their local property tax bill comes. Most cities and towns live in the stone age when it comes to the processes for their meetings. It’s like the Internet never even existed. Word documents scanned as images then turned into PDFs that require OCR are state of the art:

I’ve been working on a set of tools to collect meeting minutes, agendas and reports from hundreds of cities and towns in Vermont. We have over 150,000 documents now. We’re doing the best that we can to extract meaningful, structured data from the blobs of PDFs, Word documents and the most poorly formed HTML you’ve ever seen. We’re finding useful, interesting bits of data in this local legislative soup, Vermont Public Radio is using the information we’re finding to write stories that have been picked up by NPR and the Associated Press.

We’re never going to win the battle though. The upstream source is so polluted. We need to clean things up. I’m starting to flesh out an open source meeting management tool (Muni Meeting) that is specifically designed for Municipalities and how they run meetings. The primary benefits to a town being:

* Reduce meeting taker and organizer time
* Real time publishing of notes – zero publish time
* eDelivery of Meeting Packets (Police Officers usually hand these out manually)
* No need to convert Word docs and flatbed scanner documents to PDF
* View voting histories, profiles
* Record meeting audio via iPhone and Android apps
* Meeting topic trends
* Searchable meetings
* Low-cost or FREE tool
* Open source, open APIs for data
* Useful, structured data and information for analysis

There is a closed source, commercial vendor in this space called Granicus. They build decent tools, they have an API (limited non-public access) but they create an expensive, closed and complicated tool. It is a tool for larger cities and towns. Towns with closed circuit camera systems and $40,000,000+ budgets. Most towns in the United States have fewer than 30,000 residents. These are the towns where a Selectboard meeting might take place in a library or in a kitchen of a member. These smaller towns pass important laws and ordinances that are rarely noticed in our busy lives. Democracy is happening in public view, but we just don’t see it.

If you have gotten this far it’s likely that you’d be interested in talking with us about we’re dreaming up. I’d love to collaborate with others on this.

“My Administration is committed to creating an unprecedented level of openness in Government. We will work together to ensure the public trust and establish a system of transparency, public participation, and collaboration. Openness will strengthen our democracy and promote efficiency and effectiveness in Government.” — BARACK OBAMA

Anyone interested?

Matt MacDonald

3 ways cities can improve citizen access to online meeting information

Hi there,

I posted this over at our NearbyFYI blog and thought it would be useful to cross post it here.

At NearbyFYI we review online information and documents from hundreds of city and town websites. Our CityCrawler service has found and extracted text from over 100,000 documents for the 170+ Vermont cities and towns that we track. We’re adding new documents and municipal websites all the time and we wanted to share a few tips that make it easier for citizens to find your meeting minutes, permit forms and documents online. The information below is written for a non-technical audience but some of the changes might require assistance from your webmaster, IT department or website vendor.

Create a unique web page for each meeting

Each city or town meeting that occurs should have it’s own unique webpage for the agenda and meeting minutes. We often see cities and towns creating a single very large webpage that contains an entire year of meeting minutes. This may be convenient for the person posting the meeting minutes online but presents a number of challenges for the citizen who is trying to find a specific meeting agenda or the minutes from that meeting.

Here is an example of meeting minutes that are in a single page that requires the citizen to scroll and scroll to find what they are looking for: This long archived page structure also presents challenges to web crawlers and tools that look to create structured information from the text. Proctor, VT provides a good example for what we look for in a unique meeting minutes document. We like that this document can answer the following questions:

  1. Which town created the document? (Proctor)
  2. What type of document is this? (Meeting Minutes)
  3. Which legislative body is responsible for the document (Selectboard)
  4. When was the meeting? (November 27, 2012 – it’s better to use a full date format like this)
  5. Which board members attended the meeting? (Eric, Lloyd, Vincent, Bruce, William)

The only thing that could improve the access to this document is if it was saved as a plain text file rather than a PDF file. Creating a single web page or document for each meeting means that citizens don’t have to scan very large documents to find what they are looking for.

Save PDFs as text not images

After running our CityCrawler for several months it’s clear that cities and towns love the PDF file format to share information online. While PDF files can be a quick way to post information online, cities are often publishing documents that are scanned images which is no better than taking a photo of the document and posting it on Instagram.

The challenge here is that search engines must use OCR Optical Character Recognition software to try and extract the text from the image. Anything that makes it harder for search engines to index your documents means that fewer citizens are going to find your published information. At NearbyFYI we often see documents that could easily be saved as a text PDF but are scanned as images. Here is how this document could easily be converted to a format that search engines can use.

Steps to convert your PDF to text

  1. Scan the meeting minutes document and save it as a PDF.
  2. Open the scanned document in Adobe Acrobat.
  3. Select “File> Export> Text> Text Plain.”
  4. Name the document and click “Save.”
  5. Open the saved file and review for conversion errors.
  6. Save the corrected document and post to your website.

Allow web crawlers in Robots.txt

Most information online is found via search tools like Google or Bing. Google uses what is called a web crawler or web page indexer to review your website documents so that they can add your content to their search index. This is a good thing, you want the search companies to find your content as it’s likely the way most citizens are going to look for information about your community.

Robots.txt files contain a simple set of rules that web crawlers follow. Some websites are setup to allow crawlers others aren’t. This is an example of a robots.txt file from a city who’s online information won’t be found with a Google Search:

User-agent: *
 Disallow: /

What this means is that when a citizen searches Google for “how to get a zoning permit in pownal, vt ” they won’t find this page: Ensuring that web crawlers can access the documents you post on your website is likely the simplest thing that you can do to improve citizen access.


Jason and I have been working on CityCrawler so that we can extract structured information and provide an API to the public documents and data that our cities and towns create each day. We’ve encountered a number of other issues with city and town websites that make it harder for both citizens and web crawlers to get access to this online information and we think that we can help.

If you are interested in learning how your city can improve access to meeting minutes and other public documents please contact us at

Matt & Jason

What if adding more money to our school budget doesn’t help our children as much as we hope?

Someone questioned why I posted an article from The Onion in the Watertown Residents for Strong Schools Facebook group. Thought it might be useful to cross post it here.


I was hoping that posting the satirical article from The Onion about our education crisis might prompt further discussion. Plus it was just damn funny and I didn’t see the harm in bringing a little levity to the seriousness of the discussion.

When I read The Onion article it reminded me of a common topic on this group about how much money the school department should receive and also of a segment I saw recently on the TODAY Show about Unschooling.

Dialog in this Facebook Group reinforces the message the media covers at the national level that our current public education system is no longer producing the results we expect. Many of our conversations in this group have focused around increasing the funding our school department receives from the town and by doing so we will better support our students and teachers.

This morning I Googled, “does increasing spending on education help?” and you land on this article which does a pretty great job of dealing with the question: Basically – just spending more money won’t help as much as we think.

Maybe we should ask the tough question:
What if adding more money to our school budget doesn’t help our children as much as we hope?

When I met with former Superintendent Ann Koufman last spring she handed me a book called Rethinking Education in the Age of Technology: which spends the first few chapters describing how our current educational system came to be. Reading how our industrial revolution shaped the education system was enlightening – it also makes it clear that the 150 year old structure of our education system is ripe for radical transformation.

Self directed learning is so easy now – with just a few mouse clicks you can learn more about just about any topic on the Internet, you can attend FREE in-person computer programming classes: or attend university courses: The Khan Academy is educating millions. TED talks inspire us to learn more about our world and spark conversations and jumping off points for learning: Go to a Maker Faire with your kids: or better yet start one.

When you examine our town budget and after you add in the town appropriation, health care, pension and state and federal grants you see that our school department budget is north of $50,000,000 per year. With our 2,800 students that comes in near $18,000 per student. What if all of that money, and I mean all of it went to teachers/facilitators that interact with our kids? Imagine a world in which each educator could earn $108,000 per year to work with only 6 kids?

Could we become more involved in educating both ourselves and our children if we shook up the institution? Could we better engage with our local scientists, artists, engineers, bakers, welders and electricians if we didn’t carry the baggage of a 150 year old institution?

I believe that our government should fund the education of it’s citizens, it benefits all of us to have a more engaged and informed society. I just want to know – can we do it better and more efficiently.


He Said, She Said – Extracting data from 5 years of Watertown Town Council meetings


I think I found a way to to raise the visibility of some pretty boring civic data and present it to the public in a more useful manner. Like most cities and town Watertown, MA keeps detailed meeting minutes for each Town Council meeting that takes place. Since 2006 those meeting notes have been placed in the Document Center as PDFs. I doubt that many people are reviewing those notes, I’d honestly be surprised if there was anyone else but me looking through them. If you do review them, please say hi in the comments! Check out He Said, She Said if you want to skip the details and get to the data extracted from 5 years of Town Council meetings.

Project Goal

For this project I wanted to see if I could take the very dry and challenging to read meeting minutes from the Town Council meetings and present them in a way that might make it more interesting to people in Watertown. I’ll consider this project a success if just a handful of people explore some of these meeting minutes.


Once again my primary challenge is working with information that is locked up in PDF documents. So what could I do to take these vanilla, boring presentations of our civic employees and create a more engaging experience.

Data for this project

Tools used in this project

  • We’ll need the trusty Typhoeus and nifty Nokogiri  ruby gems to help us screen scrape and download the PDFs
  • The wonderful pdftotext tool to liberate all that juicy information from the PDF
  • Open Calais will help us find interesting people, quotes and business mentioned in the minutes and we will use the Calais Ruby gem to help us here.
  • A number of ruby scripts to use all these fine tools
  • Ruby on Rails running on Heroku will serve up the now super awesome meeting notes

The Result?

He Said, She Said is the website that shows quotes from the Town Council meetings. I’m pretty happy. I was able to pull down the PDFs, convert them to text, run them through the Open Calais web service and present them in a Rails app in a much more compelling way. The basic website that shows quotes from Town Council meetings is called He Said, She Said (thanks for the name Kimmi!) I’ve found it a fun way to look at quotes and dive into older meetings. There is obviously much more that should be done but I’d like to get it into the hands of more people in Watertown to collect feedback.

“Councilor Lawn indicated that a few business owners contacted him and stated that they would be willing to pay more for the service than lose the service altogether.”

Steps to reproduce this project in your town

  1. Locate your town meeting notes. Hopefully they are online and if you are lucky they are at least in a PDF format.
  2. Download one PDF and try to use the scripts and tools mentioned to extract the text. You may even want to copy the text from the PDF to try the Open Calais Viewer. Using their free viewer, you’ll get a sense of how useful their entity extraction is before you commit all the way.
  3. If Open Calais seems to provide useful results for your town meeting notes you can use, modify or improve the Ruby script (town-council-parse.rb) that I have provided, to screen scrape your town website and pull down the PDFs for you to locally work with them.
  4. After you have downloaded the PDFs using the script you’ll need to convert the PDFs to a plain text format so Open Calais can extract meaning from them. Take a look at the convert_to_pdf.rb script. It is very, very simple as it just converts all the PDFs it finds in your current directory into text files. If pdftotext can do batch jobs I didn’t find it right away. After running this script you should see a .txt file with the same name as the pdf.
  5. Now after you have your text files you’ll need to send the text that was extracted from the PDF to the Open Calais web service for them to generate meaning and structure from your documents. The script, extract-entities.rb uses the Calais Ruby gem to make our requests much easier. When running this script it will take a while to generate the files from the plain text documents. I like working with the JSON data format so that is what we get back from Open Calais.
  6. JSON isn’t the most user friendly format for people to look at so we should do something about that. How about we convert it to the more Excel and lay person friendly format of Comma Separated Values, CSV. Using the Siren Ruby gem we use create-quotes-csv.rb to parse the JSON returned and stored in our new csv files.
  7. At this point in your file system you should have 3 files for each meeting. The PDF, the .txt file and the .txt.json file. You should also have a new file called quotes.csv that provides a CSV file containing all of the quotes from the meetings that Open Calais located.
  8. Upload the PDFs to Scribd so people can more easily read them on the web. I currently have moved over 93 of the 126 documents before I hit the Scribd rate limit. Hopefully I will remember to go back and add the remaining documents. For my own reminder Scribd shut me off at TC%20Minutes%204.24.2007.pdf.
  9. You may find this is a place where you might stop. Having the CSV file of quotes gives you quite a bit of ability to store it in a relational database and perform fun interesting queries. I wanted to provide a more visible face to this information so I used Heroku to run a very, very simple Rails server that only currently shows a single random quote from the meetings.

Next Steps

I have known about ScraperWiki for a while but haven’t yet really tried it out. I’d like to see if I can migrate the collection of scripts that I’ve been using over to that tool. I also need to finish uploading documents


Some of the scripts

[gist id=1176346]

Mapping the location of building permit submissions in Watertown, MA


I’m trying weekly to identify and hack on data that is published in the Watertown, MA document center in order to provide it in a manner that people might find more useful than a PDF, Excel or PowerPoint presentation. Today I decided to focus some effort on the building permits. We’re about to do some water damage work in my daughters room so this felt like a relevant area to hack on.

Hack Goal

Determine the effort involved in creating an automated process to place issued building permits from Watertown, MA on a google map.

Data for this hack

Tools used in this hack

My Hack Results and a description

Overall I’m pretty happy with how this hack turned out. I was able to take a very boring presentation of this pretty interesting information that was locked up in a PDF and display it on a map. Here are the hoops that I had to jump through to get this working.

  1. Download the PDF from the Watertown, MA website.
  2. Open up DeskUNPDF and use their conversion tool to identify tabular data in a PDF so it could be extracted as a comma separated value (CSV) file.
  3. Write a little Ruby script that further cleaned up the data in preparation for generating the latitude and longitude coordinates for use in the Google Geocoding API. After getting the lat/long coordinates from Google I then had to write it out as a new CSV file for Socrata to take over.
  4. Upload a new data set to Socrata and mark the lat/long fields into a location field
  5. Use Socrata to generate the Google Map view

How to improve this


  1. Automate the downloading of building permit PDFs from the Watertown website
  2. Detect when new PDFs are available for download
  3. Script the PDF to CSV conversion using the DeskUNPDF command line interface rather than the UI
  4. Automate the updating of the dataset on Socrata

Better data to start with

Extracting this data from a PDF is something that we shouldn’t have to do. This data resides in a computer database somewhere so why should we need to write scripts to scrape and cobble this data together. If you feel like joining me in writing to Ken Thompson, CBO, LCS Inspector of Buildings at to see if there is a better way to get this information I’d be grateful.

Data integrity

One major sticking point that I have is that DeskUNPDF isn’t picking up the first two rows in each PDF so I need to ask them if they know what might be going on with that. Missing the first two rows isn’t a huge deal but I’d like the dataset to be accurate.


While I think seeing the individual permits on a map is better than looking at a list in a PDF, I would like to see this data in graph form with permits plotted over time. Obviously summer months are high for permits but I would be quite interested to see how weeks and months compare with previous years.


Not a ton of code here but you can find the Ruby script below. If others are interested in helping me write scrapers for this data the scripts will be updated in github.

[gist id=1169914]


Series: How to replace your city or town website and improve it for free

This is the first in a series of posts that will evaluate the Watertown, MA website, platforms like CivicPlus and offer suggestions for methods to improve online services and reduce costs for citizens.

Union contracts, local government budgets, school district shortfalls, pensions and dealing with debt pulled me in.

We purchased a home here in the spring of 2010 and as a new home owner with one kid in the school district and another coming soon, these were topics that I wanted to learn more about. Over the past 10 months the Watertown, MA town website has become a weekly destination, where I look for budget, personnel, meeting notes and other documents. After spending significant time with the official town website I have thoughts about how the online services provided by the town could be modified to better support the citizens of Watertown.

What gives me the qualifications to evaluate the town website? Well I live in Watertown, pay taxes, I’m a software developer by trade and I’ve been in the profession of creating high end web and mobile projects since 1997. I spend all of my work hours on the web, using online tools and services to help with my job. I also read quite a bit about user experience, user centered design and visual information design to keep up to date on trends and techniques so that I can apply them to my work. I am also very interested in local government and understanding how it operates here in Watertown, MA.

Watertown has a mildly useful although challenging to navigate, website developed on the CivicPlus e-government platform. The features offered on the town website would be great if this was 2001, but a decade later, features like a content management system, document repository, events calendar, email notifications and a staff directory are commodities that a collection of free online tools can provide.

CivicPlus developed the Watertown website and they build websites that are solely focused on the government sector. They have stated that 800 cities and towns use their products. With ~87,000 municipalities there are a number of competitors that have developed products similar to CivicPlus. CivicLogic and eGov Strategies are two of the more popular but they all sell themselves by touting industry expertise and knowledge:

We know government. We know how government works. And we know how it can work better. CivicPlus transforms standard sites into powerful communications hubs.


Your government entity is unique, therefore your business challenges require a team that will create distinctive solution strategies with you in mind. At eGov Strategies, we provide technological and real-world expertise so that you can achieve success; measured by the delivery of customized services to your customers, 24×7.

The website features that each of these e-government companies offer are now baseline, table stakes for any modern free, open source web platform like WordPress, Drupal, Joomla or as plugins to web frameworks like Ruby on Rails or Django. E-government web platforms have features that are now commodities and those companies now must market unique knowledge of the municipal industry to set them apart from thousands of other web development companies.

The problem with this marketing technique is that most cities and towns face the same problem that any group or organization has when they want to share information and collaborate with each other. This is not a problem that is unique to government.

I am writing this in hopes that I can convince employees in Watertown that there are alternatives to CivicPlus that are free and can provide better engagement with citizens than the commercial e-government solutions. I also hope that a few people living in one of the ~87,000 local municipalities in the United States stumbles their way to this guide and attempts to try this.

What gives me hope is that the city of Manor, Texas has already done most of it.

Part 1: The competition

We should understand exactly what the current state of the Watertown website is so the first post in this series will review the website product from CivicPlus called CivicPlus GCMS. Watertown uses this product so I’ll use them as the benchmark for identifying issues and offering alternative, free solutions.

Let’s review the CivicPlus website marketing materials that describe why they provide the best government website solution and what features they offer.

The CivicPlus Difference

1. Helping Communities Engage & Interact

Many companies provide municipal websites. But CivicPlus does more. We create community engagement tools. Our online solutions have the power to transform the way your community does business.

2. Save Money

Let’s go straight to the bottom line. With CivicPlus services, you will save time, budget dollars and — by moving more government paperwork online — help protect the environment. We have budget-savings solutions for all types and sizes of cities and counties.

3. Work with Experts

CivicPlus has more than a decade of expertise in government best practices. We’ve launched more than 800 municipal websites for cities, counties, associations, school districts and colleges. Visit our portfolio to learn more.

4. Affordable Innovation

The overall value we offer as opposed to our competitors is truly superior. Compare both upfront and five-year costs and discover the difference. Our modules make day-to-day functions streamlined. And we provide new modules and upgrades free.

5. Update Info with Ease

Our Content Management System is easy to use. It is an intuitive interface that allows staff members to update and maintain your site in just minutes a day. Delegate user access to individuals or a group. A dynamic, up-to-the-minute site is yours. Go here to learn more.

6. Be Secure

Enjoy the security and efficiency of cloud computing. All sites are hosted within our secure facility in Kansas City, Missouri. You are assured 99.9% uptime and full data protection.

7. Power Up

When you partner with CivicPlus, you join a team of government website experts dedicated to the absolute success of your site and your community. Our work results in cost savings, efficiencies, end user benefits and industry recognition. Since 2008, CivicPlus clients have won more than 150 prominent website awards. That’s good for you and your community’s progress.

My Summary

Reviewing each of their core differentiators:

  1. Marketing Fluff: The first bullet about helping communities engage and connect is pure marketing fluff without measurable metrics so there really isn’t anything to say here.
  2. Saving Money: I’ve been to numerous town council and school committee meetings over the past 10 months. I don’t see less paper floating around the gallery – nearly all the documents presented in the Document Center are printed as handouts to the audience. I say this is straight B.S. the environment isn’t being helped by having PDFs up in the Document Center.
  3. Experts: There is nothing unique about a knowledge sharing and CMS driven information website. When I read this it says… We use our relationships with city and town managers to build a customer base. Also kudos for managing 800 clients that is nothing to sneeze at but it isn’t the 50,000,000 websites that has.
  4. Affordable: The second time cost is mentioned so this must be important to municipalities. They talk about comparing costs to the competition but maybe they don’t think that free solutions are competition? Just how much does CivicPlus cost? Well, John McKown wrote an article that shows it can cost nearly $27,000 for a three year contract. Given that the annual Watertown budget is ~$100,000,000 this is a drop in the bucket but as we look to trim everywhere, I’m planning on writing to the Town Manager to understand the full cost to Watertown.
  5. Update information: A great feature in 2001.
  6. Be Secure: Nice that the town doesn’t have to run it’s own hardware to host the website. Only one data center that is centrally located? What does the backup and redundancy plan look like when that data center goes offline?
  7. Power Up: This sounds great! How are those cost savings passed on to the citizens of the town? I respect the profession but building your own custom CMS when so many well tested, scalable and proven tools exist why not build on top of them?

Core Features

I’ve spent several months using the Watertown website so I’m intimately familiar with the various features that CivicPlus is providing that the town is using. If I have missed some features I’d love for either a Watertown or a CivicPlus employee to catch my mistakes and add a comment.

  • A Content Management System or a CMS allows town employees to update pages of information, upload documents, photos and change text on the website without having to pay expensive web developers to make the changes. This feature keeps people informed with more recent information and allows individual employees to make changes without having all text changed centrally. Pages of information are usually created with a CMS and provide a method for linking to other pages or websites.
  • The Calendar of Events provides information about meetings, concerts, recreation department events. Generally anything that you might think to put on a calendar so you can be reminded about it. Watertown happens to break their events into a few useful categories like Recreation, Town Calendar and Senior Events.
  • The Document Center is where meeting minutes, agendas, forms, reports and other documents can be stored so that people using their computers at home can read them without having to go to the town office. Most of the documents are stored as PDFs, Microsoft Word, PowerPoint or Excel files.
  • The Resource Directory contains names, phone numbers, departments and email addresses for town employees. You can search for a particular person if you happen to know their name.
  • The Emergency Alert system provides a graphic on the town website that changes color when something important happens. You can also be notified by email using the Notify Me function.
  • Notify Me provides a way for people to sign up to email distribution lists and have either an email or an SMS/text message sent when the list moderator sends a new message.
  • For a fee each CivicPlus website allows for a city or town to have a Custom Design using images, colors and fonts that provide a more unique visual design.
  • A Mobile Website Template provides easier accessibility when using mobile devices like smartphones. I don’t believe this is enabled on the Watertown website.
  • Structured Job Postings can be created and posted where visitors can sort and view job listings.

I believe I have accurately described how Watertown is currently using the CivicPlus platform and captured the key differentiators as CivicPlus sees them. If you live in Watertown, maybe you found a new feature on the town website that you were unfamiliar with. If you have thoughts or comments about your experience with the town website please add a comment to this post.

Next up

I’ll be reviewing each of the features of the Watertown website as described above and offering suggestions for improvements and alternatives.


How to get twitter alerts from your city or town


I wanted to solve a problem that I saw with how the town of Watertown, MA shares alerts with the public. Watertown uses a website tool called CivicPlus to manage their online presence. The tool has an alert system that allows town employees to send out notifications about various goings on in town via email and SMS. I think this is a great service but with the point-to-point communication of email and text messaging it felt like it was missing a public broadcast method.

Alerts that are sent via a private list-serv and SMS have no public history and are not searchable by search engines. I wanted to create a method for those alerts to reach a larger audience and so decided to make the alerts available as Tweets on Twitter. The @WatertownAlerts account on Twitter will tweet the alerts that are sent out from the town. I’ve tested the flow but am waiting for real alerts from Watertown to start flowing in.

There is nothing magical here and I’m clearly not the first to setup an automated tweet workflow but I thought it might be useful for other cities and towns using CivicPlus to know how they can create a wider distribution for their alerts.

  • Step 1: Register for the alerts on the Watertown website “Notify Me” service
  • Step 2: Enter your email address, select the alerts you want and confirm the list-serv email sent from the town list-serv
  • Step 3: Create a new Twitter account for the tweets (@WatertownAlerts)
  • Step 4: Crete a new TweetyMail account using your email address that you entered in Step 2
  • Step 5: Confirm your TweetyMail account
  • Step 6: Create a filter in your email system (I used GMail) that looks for keywords (From: civicplus Contains: Watertown)
  • Step 7: Configure your new filter to forward alert emails to TweetyMail (

When email alerts are sent by your city or town tweets should now start flowing from the list-serv to the Twitter account that you setup.

UPDATE: @WatertownAlerts posted the first alert Tweet from the town!

Watertown Recreation Fall Fun: Registration Begins Monday, August 22 View this in your browser Watertown Recreation…