There ain’t no such thing as a free (open government data) lunch

There's no such thing as a free lunch.

Credit: Dana Fradon

There ain’t no such thing as a free lunch — I like that saying. It’s important to remember that it costs money to provide access to information, even public government or municipal information. Someone must pay. But who pays and how much? I like Philip Ashlock’s Tweets so I follow him on Twitter, he also has a nifty proposal called DemocracyMap in the Knight News Challenge. Our NearbyFYI proposal is also in the semi-final round with Philip’s, so when I saw his recent Tweet about “Paying for Public Data” it piqued my interest.

“There are certain elements of our democratic system of government that are so essential to its freedoms and principles that we have to make them as accessible as possible and provide them free of charge.”

That line got my attention and guaranteed that I’d read his entire article. I realize the piece is mostly a response to David Eave’s Tech President post about grant funded projects potentially destabilizing for-profit organizations. I’m not going to weigh in on that discussion as I don’t have strong opinions, but I do want to unpack Philip’s statement that some government information should be free of charge. It’s something that I’ve written about before but I think it’s worth teasing out more.

The government data ecosystem

I’ve spent a lot of time thinking, reading and doing in the Open Government space and I think the following four statements pretty accurately represent the current government data ecosystem:

  1. Citizens must have access to information about their municipality or government.
  2. Access to information has a cost.
  3. Corporations derive revenue from government data.
  4. Citizens pay corporations for services built with government data.

I’m going to focus on statement #2, I have plenty of thoughts about #1, #3 and #4 for a later time, let’s focus on #2 first.

Access to government information has a cost

There are real costs when we interact with our government and request access to information about it. In the non-digital world we need government employees to pull documents from filing cabinets, answer phone calls and handle in-person information requests. Municipal employees attend public meetings, give presentations and summarize reports for us. The human cost to access analog government information is real.

The Open Government movement usually focuses on the digital world, but digital access to government information has real costs too. Proprietary software solutions require licensing fees and support contracts, and open source approach requires knowledgable IT staff, dev-ops and probably developers. There are support costs for open source too, people to help answer questions. There are costs to digitize documents, setup new data publishing workflows and smaller fees for servers (cloud or not), bandwidth, and electricity.

Who is paying the costs now?

I think we often miss an important point when we talk about free, public or open government data — that when we require our government to provide better access to information that it creates a new expense line item. This means that we as citizens pay for improved access — but are we getting a good return on that investment and expense? Larger cities currently pay vendors to provide Open Data portals and information publishing solutions for them. There are open source alternatives like CKAN too. These solutions can work in larger metropolitan areas where the benefits of sharing open data can be realized downstream, but smaller communities will have a harder time justifying the expense. What new open data derived services in a town of 3,000 people will be created that the community values enough to offset the expense?

I live in the Boston area and when the MBTA opened up transit data my commute improved as Google and others stepped in, using tax payer funded data to provide us with useful services based on that data. My experience with the MBTA improved as a result of that investment from my tax dollars. I think my tax dollars are being used wisely. I live in Watertown, Massachusetts (pop. 31,915) and if the town decided to use a vendor like Socrata to provide open data, a very small portion of my property tax dollars would be used to pay for that service. I’ve spent a long time thinking about what services would be created from an Open Data portal in a community of this size and I haven’t been able to think of one that would be worth our tax dollars. When budgets are being trimmed, and it comes down to more teachers or more open data portals, the teachers are going to win and probably should.

My hometown of Millinocket, Maine (pop. 4,506) and Watertown, Massachusetts are probably more like your town than Boston is. Millinocket doesn’t have a public transit system or other large scale services, so it’s harder to explain how Open Government, specifically “Open Data” will help them. When 80% of our towns have fewer than 10,000 people living in them, we should be looking at different models and incentives to get improved municipal information access. Without economic incentives, meaning cost savings — not a new expense, smaller communities will be hard pressed to adopt Open Data tools.

There is another information cost that isn’t often highlighted, I am certainly paying a portion of  a City Clerk’s time when an RFP bid monitoring company from out of state calls my town office and takes 5 minutes to ask for information about new bids. A company that does not pay property taxes in my town. When a fellow citizen emails the Town Manager to ask a budget question it takes time for them to find the information and provide a response. I understand that these examples result in a very tiny portion of my tax bill, but I think it is important to highlight that we’re already paying for data access, it’s just poor quality and inefficient.

So, wait who should pay then?

The Internet is littered with services and software vendors that cost tax payers money, where local government is the customer and our tax dollars help their businesses profit. Code for America references a GovWin report that states $60B will be spent on IT by local and state government in 2013. There is a long list of companies suckling from that wealthy government IT teat. From simple website vendors like CivicPlus, GovOffice and Virtual Town Hall to older established companies like Tyler Technologies, IBM and Microsoft. There are thousands of small mom and pop vendors, mid-tier $5M-$20M companies and fortune 500s that provide solutions for building permitting, parking tickets, accounting and payroll. All of these solutions cost us as tax payers money.

So, you may be wondering how do we get access to municipal information if we’re not paying these companies to provide tools and services? Good question.

Companies like BidClerk.com, CrimeReports.com and eRepublic broker access to municipal information to other companies. They recognized value in the data that our cities and towns generate and spent time and money building up methods to collect this difficult to access information. The Adam and Eve of this approach are Westlaw and Lexis Nexis. I know that there are serious, important discussions taking place about these types of companies and their practice of paywalling access to important documents. To be clear, I believe citizens MUST have access to these documents and information without having to pay a second time. What is important though is that these companies figured out that there are other entities that derive business value from having access to this type of information. What Westlaw, LexisNexis and until only recently JSTOR, couldn’t figure out is how to broker access to data for those that derive business value, while providing access to the public.

To re-appropriate William Gibson’s famous quote “Access to Government data is already here — it’s just not very evenly distributed.”

What are we proposing that is different then?

The approach that we promoting at NearbyFYI blends both of these realities, that there are companies that derive value from better government data and that citizens must have access to it. Access to information has a cost, we just want to shift it over to those that benefit financially in a more efficient way. We see a better way to get high quality data into the hands of those companies willing to pay for it by providing better tools and services for our government. We don’t see expensive “Open Data Portals” as being that solution. We’re going to treat the government as a user, not a customer, helping government employees do their jobs more efficiently. We see a wealth of valuable “dark data” in our municipalities, data that other companies will pay to get access to, we just need to give municipalities better tools to provide it.

Thanks,
Matt

 

Local government is sexy

Hi,

Local government is sexy.

It isn’t really, but it could be made much simpler to understand. Most of the work that takes place in local government is done in Select Boards, Town Councils and Sub-Committees like Finance and Public Works. Most people could care less what goes on these meetings until their local property tax bill comes. Most cities and towns live in the stone age when it comes to the processes for their meetings. It’s like the Internet never even existed. Word documents scanned as images then turned into PDFs that require OCR are state of the art: http://www.cityofwestfield.org/Files/AgendaCenter/Agendas/68/Archives/57/01-08-13%20Conservation%20PH%20Niemiec.pdf.

I’ve been working on a set of tools to collect meeting minutes, agendas and reports from hundreds of cities and towns in Vermont. We have over 150,000 documents now. We’re doing the best that we can to extract meaningful, structured data from the blobs of PDFs, Word documents and the most poorly formed HTML you’ve ever seen. We’re finding useful, interesting bits of data in this local legislative soup, Vermont Public Radio is using the information we’re finding to write stories that have been picked up by NPR and the Associated Press.

We’re never going to win the battle though. The upstream source is so polluted. We need to clean things up. I’m starting to flesh out an open source meeting management tool (Muni Meeting) that is specifically designed for Municipalities and how they run meetings. The primary benefits to a town being:

* Reduce meeting taker and organizer time
* Real time publishing of notes – zero publish time
* eDelivery of Meeting Packets (Police Officers usually hand these out manually)
* No need to convert Word docs and flatbed scanner documents to PDF
* View voting histories, profiles
* Record meeting audio via iPhone and Android apps
* Meeting topic trends
* Searchable meetings
* Low-cost or FREE tool
* Open source, open APIs for data
* Useful, structured data and information for analysis

There is a closed source, commercial vendor in this space called Granicus. They build decent tools, they have an API (limited non-public access) but they create an expensive, closed and complicated tool. It is a tool for larger cities and towns. Towns with closed circuit camera systems and $40,000,000+ budgets. Most towns in the United States have fewer than 30,000 residents. These are the towns where a Selectboard meeting might take place in a library or in a kitchen of a member. These smaller towns pass important laws and ordinances that are rarely noticed in our busy lives. Democracy is happening in public view, but we just don’t see it.

If you have gotten this far it’s likely that you’d be interested in talking with us about we’re dreaming up. I’d love to collaborate with others on this.

“My Administration is committed to creating an unprecedented level of openness in Government. We will work together to ensure the public trust and establish a system of transparency, public participation, and collaboration. Openness will strengthen our democracy and promote efficiency and effectiveness in Government.” — BARACK OBAMA

http://www.whitehouse.gov/the_press_office/TransparencyandOpenGovernment

Anyone interested? matt.macdonald@gmail.com.

Thanks,
Matt MacDonald

3 ways cities can improve citizen access to online meeting information

Hi there,

I posted this over at our NearbyFYI blog and thought it would be useful to cross post it here.

At NearbyFYI we review online information and documents from hundreds of city and town websites. Our CityCrawler service has found and extracted text from over 100,000 documents for the 170+ Vermont cities and towns that we track. We’re adding new documents and municipal websites all the time and we wanted to share a few tips that make it easier for citizens to find your meeting minutes, permit forms and documents online. The information below is written for a non-technical audience but some of the changes might require assistance from your webmaster, IT department or website vendor.

Create a unique web page for each meeting

Each city or town meeting that occurs should have it’s own unique webpage for the agenda and meeting minutes. We often see cities and towns creating a single very large webpage that contains an entire year of meeting minutes. This may be convenient for the person posting the meeting minutes online but presents a number of challenges for the citizen who is trying to find a specific meeting agenda or the minutes from that meeting.

Here is an example of meeting minutes that are in a single page that requires the citizen to scroll and scroll to find what they are looking for: http://www.shrewsburyvt.org/sbminsarchive.php This long archived page structure also presents challenges to web crawlers and tools that look to create structured information from the text. Proctor, VT provides a good example for what we look for in a unique meeting minutes document. We like that this document can answer the following questions:

  1. Which town created the document? (Proctor)
  2. What type of document is this? (Meeting Minutes)
  3. Which legislative body is responsible for the document (Selectboard)
  4. When was the meeting? (November 27, 2012 – it’s better to use a full date format like this)
  5. Which board members attended the meeting? (Eric, Lloyd, Vincent, Bruce, William)

The only thing that could improve the access to this document is if it was saved as a plain text file rather than a PDF file. Creating a single web page or document for each meeting means that citizens don’t have to scan very large documents to find what they are looking for.

Save PDFs as text not images

After running our CityCrawler for several months it’s clear that cities and towns love the PDF file format to share information online. While PDF files can be a quick way to post information online, cities are often publishing documents that are scanned images which is no better than taking a photo of the document and posting it on Instagram.

The challenge here is that search engines must use OCR Optical Character Recognition software to try and extract the text from the image. Anything that makes it harder for search engines to index your documents means that fewer citizens are going to find your published information. At NearbyFYI we often see documents that could easily be saved as a text PDF but are scanned as images. Here is how this document could easily be converted to a format that search engines can use.

Steps to convert your PDF to text

  1. Scan the meeting minutes document and save it as a PDF.
  2. Open the scanned document in Adobe Acrobat.
  3. Select “File> Export> Text> Text Plain.”
  4. Name the document and click “Save.”
  5. Open the saved file and review for conversion errors.
  6. Save the corrected document and post to your website.

http://www.ehow.com/how4854208convert-pdf-document-text-file.html

Allow web crawlers in Robots.txt

Most information online is found via search tools like Google or Bing. Google uses what is called a web crawler or web page indexer to review your website documents so that they can add your content to their search index. This is a good thing, you want the search companies to find your content as it’s likely the way most citizens are going to look for information about your community.

Robots.txt files contain a simple set of rules that web crawlers follow. Some websites are setup to allow crawlers others aren’t. This is an example of a robots.txt file from a city who’s online information won’t be found with a Google Search:

User-agent: *
 Disallow: /

What this means is that when a citizen searches Google for “how to get a zoning permit in pownal, vt ” they won’t find this page: http://www.pownalvt.org/planning-commission/town-plan/zoning-rules/zoning-permit-application/. Ensuring that web crawlers can access the documents you post on your website is likely the simplest thing that you can do to improve citizen access.

Summary

Jason and I have been working on CityCrawler so that we can extract structured information and provide an API to the public documents and data that our cities and towns create each day. We’ve encountered a number of other issues with city and town websites that make it harder for both citizens and web crawlers to get access to this online information and we think that we can help.

If you are interested in learning how your city can improve access to meeting minutes and other public documents please contact us at info@nearbyfyi.com.

Thanks,
Matt & Jason

What if adding more money to our school budget doesn’t help our children as much as we hope?

Someone questioned why I posted an article from The Onion in the Watertown Residents for Strong Schools Facebook group. Thought it might be useful to cross post it here.

Hi,

I was hoping that posting the satirical article from The Onion about our education crisis might prompt further discussion. Plus it was just damn funny and I didn’t see the harm in bringing a little levity to the seriousness of the discussion.

When I read The Onion article it reminded me of a common topic on this group about how much money the school department should receive and also of a segment I saw recently on the TODAY Show about Unschooling.

Dialog in this Facebook Group reinforces the message the media covers at the national level http://www.educationnation.com/ http://www.pbs.org/wgbh/pages/frontline/education/ that our current public education system is no longer producing the results we expect. Many of our conversations in this group have focused around increasing the funding our school department receives from the town and by doing so we will better support our students and teachers.

This morning I Googled, “does increasing spending on education help?” and you land on this article which does a pretty great job of dealing with the question: http://www.thefiscaltimes.com/Articles/2011/06/06/School-Budgets-The-Worst-Education-Money-Can-Buy.aspx#page1 Basically – just spending more money won’t help as much as we think.

Maybe we should ask the tough question:
What if adding more money to our school budget doesn’t help our children as much as we hope?

When I met with former Superintendent Ann Koufman last spring she handed me a book called Rethinking Education in the Age of Technology: http://www.amazon.com/Rethinking-Education-Technology-Education-Connections-Education-Connections/dp/0807750026 which spends the first few chapters describing how our current educational system came to be. Reading how our industrial revolution shaped the education system was enlightening – it also makes it clear that the 150 year old structure of our education system is ripe for radical transformation.

Self directed learning is so easy now – with just a few mouse clicks you can learn more about just about any topic on the Internet, you can attend FREE in-person computer programming classes: http://meetup.bostonpython.com/events/17433132/?eventId=17433132&action=detail or attend university courses: https://www.ai-class.com/. The Khan Academy http://www.khanacademy.org/ is educating millions. TED talks inspire us to learn more about our world and spark conversations and jumping off points for learning: http://www.ted.com/. Go to a Maker Faire with your kids: http://makerfaire.com/ or better yet start one.

When you examine our town budget and after you add in the town appropriation, health care, pension and state and federal grants you see that our school department budget is north of $50,000,000 per year. With our 2,800 students that comes in near $18,000 per student. What if all of that money, and I mean all of it went to teachers/facilitators that interact with our kids? Imagine a world in which each educator could earn $108,000 per year to work with only 6 kids?

Could we become more involved in educating both ourselves and our children if we shook up the institution? Could we better engage with our local scientists, artists, engineers, bakers, welders and electricians if we didn’t carry the baggage of a 150 year old institution?

I believe that our government should fund the education of it’s citizens, it benefits all of us to have a more engaged and informed society. I just want to know – can we do it better and more efficiently.

Thanks,
Matt

He Said, She Said – Extracting data from 5 years of Watertown Town Council meetings

Hi,

I think I found a way to to raise the visibility of some pretty boring civic data and present it to the public in a more useful manner. Like most cities and town Watertown, MA keeps detailed meeting minutes for each Town Council meeting that takes place. Since 2006 those meeting notes have been placed in the Document Center as PDFs. I doubt that many people are reviewing those notes, I’d honestly be surprised if there was anyone else but me looking through them. If you do review them, please say hi in the comments! Check out He Said, She Said if you want to skip the details and get to the data extracted from 5 years of Town Council meetings.

Project Goal

For this project I wanted to see if I could take the very dry and challenging to read meeting minutes from the Town Council meetings and present them in a way that might make it more interesting to people in Watertown. I’ll consider this project a success if just a handful of people explore some of these meeting minutes.

Challenges

Once again my primary challenge is working with information that is locked up in PDF documents. So what could I do to take these vanilla, boring presentations of our civic employees and create a more engaging experience.

Data for this project

Tools used in this project

  • We’ll need the trusty Typhoeus and nifty Nokogiri  ruby gems to help us screen scrape and download the PDFs
  • The wonderful pdftotext tool to liberate all that juicy information from the PDF
  • Open Calais will help us find interesting people, quotes and business mentioned in the minutes and we will use the Calais Ruby gem to help us here.
  • A number of ruby scripts to use all these fine tools
  • Ruby on Rails running on Heroku will serve up the now super awesome meeting notes

The Result?

He Said, She Said is the website that shows quotes from the Town Council meetings. I’m pretty happy. I was able to pull down the PDFs, convert them to text, run them through the Open Calais web service and present them in a Rails app in a much more compelling way. The basic website that shows quotes from Town Council meetings is called He Said, She Said (thanks for the name Kimmi!) I’ve found it a fun way to look at quotes and dive into older meetings. There is obviously much more that should be done but I’d like to get it into the hands of more people in Watertown to collect feedback.

“Councilor Lawn indicated that a few business owners contacted him and stated that they would be willing to pay more for the service than lose the service altogether.”

Steps to reproduce this project in your town

  1. Locate your town meeting notes. Hopefully they are online and if you are lucky they are at least in a PDF format.
  2. Download one PDF and try to use the scripts and tools mentioned to extract the text. You may even want to copy the text from the PDF to try the Open Calais Viewer. Using their free viewer, you’ll get a sense of how useful their entity extraction is before you commit all the way.
  3. If Open Calais seems to provide useful results for your town meeting notes you can use, modify or improve the Ruby script (town-council-parse.rb) that I have provided, to screen scrape your town website and pull down the PDFs for you to locally work with them.
  4. After you have downloaded the PDFs using the script you’ll need to convert the PDFs to a plain text format so Open Calais can extract meaning from them. Take a look at the convert_to_pdf.rb script. It is very, very simple as it just converts all the PDFs it finds in your current directory into text files. If pdftotext can do batch jobs I didn’t find it right away. After running this script you should see a .txt file with the same name as the pdf.
  5. Now after you have your text files you’ll need to send the text that was extracted from the PDF to the Open Calais web service for them to generate meaning and structure from your documents. The script, extract-entities.rb uses the Calais Ruby gem to make our requests much easier. When running this script it will take a while to generate the files from the plain text documents. I like working with the JSON data format so that is what we get back from Open Calais.
  6. JSON isn’t the most user friendly format for people to look at so we should do something about that. How about we convert it to the more Excel and lay person friendly format of Comma Separated Values, CSV. Using the Siren Ruby gem we use create-quotes-csv.rb to parse the JSON returned and stored in our new csv files.
  7. At this point in your file system you should have 3 files for each meeting. The PDF, the .txt file and the .txt.json file. You should also have a new file called quotes.csv that provides a CSV file containing all of the quotes from the meetings that Open Calais located.
  8. Upload the PDFs to Scribd so people can more easily read them on the web. I currently have moved over 93 of the 126 documents before I hit the Scribd rate limit. Hopefully I will remember to go back and add the remaining documents. For my own reminder Scribd shut me off at TC%20Minutes%204.24.2007.pdf.
  9. You may find this is a place where you might stop. Having the CSV file of quotes gives you quite a bit of ability to store it in a relational database and perform fun interesting queries. I wanted to provide a more visible face to this information so I used Heroku to run a very, very simple Rails server that only currently shows a single random quote from the meetings.

Next Steps

I have known about ScraperWiki for a while but haven’t yet really tried it out. I’d like to see if I can migrate the collection of scripts that I’ve been using over to that tool. I also need to finish uploading documents

Thanks,
Matt

Some of the scripts

[gist id=1176346]

Mapping the location of building permit submissions in Watertown, MA

Hi,

I’m trying weekly to identify and hack on data that is published in the Watertown, MA document center in order to provide it in a manner that people might find more useful than a PDF, Excel or PowerPoint presentation. Today I decided to focus some effort on the building permits. We’re about to do some water damage work in my daughters room so this felt like a relevant area to hack on.

Hack Goal

Determine the effort involved in creating an automated process to place issued building permits from Watertown, MA on a google map.

Data for this hack

Tools used in this hack

My Hack Results and a description

Overall I’m pretty happy with how this hack turned out. I was able to take a very boring presentation of this pretty interesting information that was locked up in a PDF and display it on a map. Here are the hoops that I had to jump through to get this working.

  1. Download the PDF from the Watertown, MA website.
  2. Open up DeskUNPDF and use their conversion tool to identify tabular data in a PDF so it could be extracted as a comma separated value (CSV) file.
  3. Write a little Ruby script that further cleaned up the data in preparation for generating the latitude and longitude coordinates for use in the Google Geocoding API. After getting the lat/long coordinates from Google I then had to write it out as a new CSV file for Socrata to take over.
  4. Upload a new data set to Socrata and mark the lat/long fields into a location field
  5. Use Socrata to generate the Google Map view

How to improve this

Automation

  1. Automate the downloading of building permit PDFs from the Watertown website
  2. Detect when new PDFs are available for download
  3. Script the PDF to CSV conversion using the DeskUNPDF command line interface rather than the UI
  4. Automate the updating of the dataset on Socrata

Better data to start with

Extracting this data from a PDF is something that we shouldn’t have to do. This data resides in a computer database somewhere so why should we need to write scripts to scrape and cobble this data together. If you feel like joining me in writing to Ken Thompson, CBO, LCS Inspector of Buildings at kthompson@watertown-ma.gov to see if there is a better way to get this information I’d be grateful.

Data integrity

One major sticking point that I have is that DeskUNPDF isn’t picking up the first two rows in each PDF so I need to ask them if they know what might be going on with that. Missing the first two rows isn’t a huge deal but I’d like the dataset to be accurate.

Trends

While I think seeing the individual permits on a map is better than looking at a list in a PDF, I would like to see this data in graph form with permits plotted over time. Obviously summer months are high for permits but I would be quite interested to see how weeks and months compare with previous years.

Code

Not a ton of code here but you can find the Ruby script below. If others are interested in helping me write scrapers for this data the scripts will be updated in github.

[gist id=1169914]

Thanks,
Matt

Series: How to replace your city or town website and improve it for free

This is the first in a series of posts that will evaluate the Watertown, MA website, platforms like CivicPlus and offer suggestions for methods to improve online services and reduce costs for citizens.

Union contracts, local government budgets, school district shortfalls, pensions and dealing with debt pulled me in.

We purchased a home here in the spring of 2010 and as a new home owner with one kid in the school district and another coming soon, these were topics that I wanted to learn more about. Over the past 10 months the Watertown, MA town website has become a weekly destination, where I look for budget, personnel, meeting notes and other documents. After spending significant time with the official town website I have thoughts about how the online services provided by the town could be modified to better support the citizens of Watertown.

What gives me the qualifications to evaluate the town website? Well I live in Watertown, pay taxes, I’m a software developer by trade and I’ve been in the profession of creating high end web and mobile projects since 1997. I spend all of my work hours on the web, using online tools and services to help with my job. I also read quite a bit about user experience, user centered design and visual information design to keep up to date on trends and techniques so that I can apply them to my work. I am also very interested in local government and understanding how it operates here in Watertown, MA.

Watertown has a mildly useful although challenging to navigate, website developed on the CivicPlus e-government platform. The features offered on the town website would be great if this was 2001, but a decade later, features like a content management system, document repository, events calendar, email notifications and a staff directory are commodities that a collection of free online tools can provide.

CivicPlus developed the Watertown website and they build websites that are solely focused on the government sector. They have stated that 800 cities and towns use their products. With ~87,000 municipalities there are a number of competitors that have developed products similar to CivicPlus. CivicLogic and eGov Strategies are two of the more popular but they all sell themselves by touting industry expertise and knowledge:

We know government. We know how government works. And we know how it can work better. CivicPlus transforms standard sites into powerful communications hubs.

or

Your government entity is unique, therefore your business challenges require a team that will create distinctive solution strategies with you in mind. At eGov Strategies, we provide technological and real-world expertise so that you can achieve success; measured by the delivery of customized services to your customers, 24×7.

The website features that each of these e-government companies offer are now baseline, table stakes for any modern free, open source web platform like WordPress, Drupal, Joomla or as plugins to web frameworks like Ruby on Rails or Django. E-government web platforms have features that are now commodities and those companies now must market unique knowledge of the municipal industry to set them apart from thousands of other web development companies.

The problem with this marketing technique is that most cities and towns face the same problem that any group or organization has when they want to share information and collaborate with each other. This is not a problem that is unique to government.

I am writing this in hopes that I can convince employees in Watertown that there are alternatives to CivicPlus that are free and can provide better engagement with citizens than the commercial e-government solutions. I also hope that a few people living in one of the ~87,000 local municipalities in the United States stumbles their way to this guide and attempts to try this.

What gives me hope is that the city of Manor, Texas has already done most of it.

Part 1: The competition

We should understand exactly what the current state of the Watertown website is so the first post in this series will review the website product from CivicPlus called CivicPlus GCMS. Watertown uses this product so I’ll use them as the benchmark for identifying issues and offering alternative, free solutions.

Let’s review the CivicPlus website marketing materials that describe why they provide the best government website solution and what features they offer.

The CivicPlus Difference

1. Helping Communities Engage & Interact

Many companies provide municipal websites. But CivicPlus does more. We create community engagement tools. Our online solutions have the power to transform the way your community does business.

2. Save Money

Let’s go straight to the bottom line. With CivicPlus services, you will save time, budget dollars and — by moving more government paperwork online — help protect the environment. We have budget-savings solutions for all types and sizes of cities and counties.

3. Work with Experts

CivicPlus has more than a decade of expertise in government best practices. We’ve launched more than 800 municipal websites for cities, counties, associations, school districts and colleges. Visit our portfolio to learn more.

4. Affordable Innovation

The overall value we offer as opposed to our competitors is truly superior. Compare both upfront and five-year costs and discover the difference. Our modules make day-to-day functions streamlined. And we provide new modules and upgrades free.

5. Update Info with Ease

Our Content Management System is easy to use. It is an intuitive interface that allows staff members to update and maintain your site in just minutes a day. Delegate user access to individuals or a group. A dynamic, up-to-the-minute site is yours. Go here to learn more.

6. Be Secure

Enjoy the security and efficiency of cloud computing. All sites are hosted within our secure facility in Kansas City, Missouri. You are assured 99.9% uptime and full data protection.

7. Power Up

When you partner with CivicPlus, you join a team of government website experts dedicated to the absolute success of your site and your community. Our work results in cost savings, efficiencies, end user benefits and industry recognition. Since 2008, CivicPlus clients have won more than 150 prominent website awards. That’s good for you and your community’s progress.

My Summary

Reviewing each of their core differentiators:

  1. Marketing Fluff: The first bullet about helping communities engage and connect is pure marketing fluff without measurable metrics so there really isn’t anything to say here.
  2. Saving Money: I’ve been to numerous town council and school committee meetings over the past 10 months. I don’t see less paper floating around the gallery – nearly all the documents presented in the Document Center are printed as handouts to the audience. I say this is straight B.S. the environment isn’t being helped by having PDFs up in the Document Center.
  3. Experts: There is nothing unique about a knowledge sharing and CMS driven information website. When I read this it says… We use our relationships with city and town managers to build a customer base. Also kudos for managing 800 clients that is nothing to sneeze at but it isn’t the 50,000,000 websites that WordPress.com has.
  4. Affordable: The second time cost is mentioned so this must be important to municipalities. They talk about comparing costs to the competition but maybe they don’t think that free solutions are competition? Just how much does CivicPlus cost? Well, John McKown wrote an article that shows it can cost nearly $27,000 for a three year contract. Given that the annual Watertown budget is ~$100,000,000 this is a drop in the bucket but as we look to trim everywhere, I’m planning on writing to the Town Manager to understand the full cost to Watertown.
  5. Update information: A great feature in 2001.
  6. Be Secure: Nice that the town doesn’t have to run it’s own hardware to host the website. Only one data center that is centrally located? What does the backup and redundancy plan look like when that data center goes offline?
  7. Power Up: This sounds great! How are those cost savings passed on to the citizens of the town? I respect the profession but building your own custom CMS when so many well tested, scalable and proven tools exist why not build on top of them?

Core Features

I’ve spent several months using the Watertown website so I’m intimately familiar with the various features that CivicPlus is providing that the town is using. If I have missed some features I’d love for either a Watertown or a CivicPlus employee to catch my mistakes and add a comment.

  • A Content Management System or a CMS allows town employees to update pages of information, upload documents, photos and change text on the website without having to pay expensive web developers to make the changes. This feature keeps people informed with more recent information and allows individual employees to make changes without having all text changed centrally. Pages of information are usually created with a CMS and provide a method for linking to other pages or websites.
  • The Calendar of Events provides information about meetings, concerts, recreation department events. Generally anything that you might think to put on a calendar so you can be reminded about it. Watertown happens to break their events into a few useful categories like Recreation, Town Calendar and Senior Events.
  • The Document Center is where meeting minutes, agendas, forms, reports and other documents can be stored so that people using their computers at home can read them without having to go to the town office. Most of the documents are stored as PDFs, Microsoft Word, PowerPoint or Excel files.
  • The Resource Directory contains names, phone numbers, departments and email addresses for town employees. You can search for a particular person if you happen to know their name.
  • The Emergency Alert system provides a graphic on the town website that changes color when something important happens. You can also be notified by email using the Notify Me function.
  • Notify Me provides a way for people to sign up to email distribution lists and have either an email or an SMS/text message sent when the list moderator sends a new message.
  • For a fee each CivicPlus website allows for a city or town to have a Custom Design using images, colors and fonts that provide a more unique visual design.
  • A Mobile Website Template provides easier accessibility when using mobile devices like smartphones. I don’t believe this is enabled on the Watertown website.
  • Structured Job Postings can be created and posted where visitors can sort and view job listings.

I believe I have accurately described how Watertown is currently using the CivicPlus platform and captured the key differentiators as CivicPlus sees them. If you live in Watertown, maybe you found a new feature on the town website that you were unfamiliar with. If you have thoughts or comments about your experience with the town website please add a comment to this post.

Next up

I’ll be reviewing each of the features of the Watertown website as described above and offering suggestions for improvements and alternatives.

Thanks,
Matt

How to get twitter alerts from your city or town

Hi,

I wanted to solve a problem that I saw with how the town of Watertown, MA shares alerts with the public. Watertown uses a website tool called CivicPlus to manage their online presence. The tool has an alert system that allows town employees to send out notifications about various goings on in town via email and SMS. I think this is a great service but with the point-to-point communication of email and text messaging it felt like it was missing a public broadcast method.

Alerts that are sent via a private list-serv and SMS have no public history and are not searchable by search engines. I wanted to create a method for those alerts to reach a larger audience and so decided to make the alerts available as Tweets on Twitter. The @WatertownAlerts account on Twitter will tweet the alerts that are sent out from the town. I’ve tested the flow but am waiting for real alerts from Watertown to start flowing in.

There is nothing magical here and I’m clearly not the first to setup an automated tweet workflow but I thought it might be useful for other cities and towns using CivicPlus to know how they can create a wider distribution for their alerts.

  • Step 1: Register for the alerts on the Watertown website “Notify Me” service
  • Step 2: Enter your email address, select the alerts you want and confirm the list-serv email sent from the town list-serv
  • Step 3: Create a new Twitter account for the tweets (@WatertownAlerts)
  • Step 4: Crete a new TweetyMail account using your email address that you entered in Step 2
  • Step 5: Confirm your TweetyMail account
  • Step 6: Create a filter in your email system (I used GMail) that looks for keywords (From: civicplus Contains: Watertown)
  • Step 7: Configure your new filter to forward alert emails to TweetyMail (post@tweetymail.com)

When email alerts are sent by your city or town tweets should now start flowing from the list-serv to the Twitter account that you setup.

UPDATE: @WatertownAlerts posted the first alert Tweet from the town!

Watertown Recreation Fall Fun: Registration Begins Monday, August 22 View this in your browser Watertown Recreation…http://t.co/IhzuGNV

Thanks,
Matt

API access questions for Tyler Technologies creator of MUNIS

Below is an email that I wrote to TylerTech in early July. Summer is nearly over so it’s time to pick back up on this.

UPDATE: I heard back a few days after I posted this and the official response is after my initial email.

Hi,

I’m a resident of Watertown, MA and a professional software developer. Watertown uses the TylerTech MUNIS system. I have an interest in my local government and helping my fellow residents get improved access to raw information and data currently stored in the MUNIS system. Currently the town auditor and school business administrators are only able to send either PDF or Excel spreadsheets of specific reports that they run.

I’m wondering if there is a more interactive method for citizen software developers to be able to query and understand the data/information that is stored in the MUNIS system.

Does the MUNIS system provide an API (Application Programming Interface) exposed as a web service in one of the more popular forms of either REST or SOAP over HTTP? If there is not currently an API and web service in place where does the priority of this feature exist on the MUNIS development roadmap?

Thanks,
Matt

TylerTech response on 08/22/2011

Thank you for your request for information.

We value our relationship with our clients and are committed to protecting their privacy and security protocols.  I have shared your request with the town and would direct you to the Town of Watertown for information regarding your request.

Regards,
Patty Leino

Patty Leino, HDI-CSR
Senior Technical Support Analyst
Tyler Technologies, Inc.

P: 800.772.2260 ext: 4152
F: 207.781.3585
www.tylertech.com

Reaching out to Matrix Consulting Group about the Watertown, MA Public Safety Study

Dear Richard Brady and Robin Haley,

My name is Matt MacDonald a resident of Watertown, MA, a town that has contracted services from Matrix Consulting Group to perform a Public Safety Study. I wanted to introduce myself with the intent of discussing interest in your methods for collecting and more importantly sharing the raw data required for this project.

I have been actively working and communicating with both Chief Deveau and Chief Orangio to retrieve raw incident data from the computer aided dispatch system. This has been an ongoing, slow and challenging process. I am a software developer by trade and have an interest in the underlying data. I believe that citizens should have access to the most granular form of the data to help them perform their own analysis and ask more detailed questions. The data that is collected in these systems is owned by the people and should not be locked up in proprietary systems.

I’m very interested to see the output of your report but would be very grateful to have a conversation about your methods for retrieving this information from the public safety departments and what level of detail you will be providing.

My notes and communications with the various Watertown, MA personnel can be found here: http://www.mattmacdonald.com/category/police/

Thank you for your time,
Matt MacDonald
34 Hardy Ave, Watertown, MA