The Ultralight Startup
The Ultralight Starup

The Ultralight Startup shows you don't need an MBA, a trust fund, or even experience running your own company to become a star in the tech world. The Ultralight Startup is a comprehensive, easy-to-follow guide that will prepare any entrepreneur for success. Learn how to raise money, launch a product, get covered on TechCrunch, and the secrets behind companies like Onswipe, Foursquare, Twitter, Dropbox, Mint, and more.

14 Jun

Excel Is The World’s Most Used “Database”

Excel Is The World’s Most Used “Database”

First off, the title is meant to get a point across.  The statement of Microsoft Excel being the world’s most used database may or may not be true, but it would be a fair statement  to claim that “An alarmingly large number of individuals use Microsoft Excel to store non-numeric arrays of information that should probably be stored in a database / be created by a simple web application”.  Despite all of the technology advancements we have today there is still no solution for the age old problem of using Microsoft Excel as a database for keeping track of information and basic businesses processes.  The logic behind this post has been in my head for years, but the inspiration for it finally came about when I was sitting in on a pitch an entrepreneur was giving to my friend Adam at True Ventures.  Part of the conversation went something like this:

Adam:  So how do they do this (process X) now?

Entrepreneur:  They keep track of it in an excel spreadsheet.

Me:  Yeah, that’s a good sign disruption is needed.  If you want to find a good SaaS app to build, just find an industry/process that is largely dominated by excel spreadsheets.

Why Is Excel Still Used Like This?

It Just Works + It’s Familiar-  Though this post seems to be “bashing” the use of Excel as a simple database, it’s still the easiest solution out there.  Fire up a new excel document, define the header columns, and start entering in data.  In the long run, Excel is a poor choice, but in the short run, it’s the easiest way to get the job done  Once you’re done, click email and everyone else gets the spreadsheet.  At this point, the headaches usually start rolling in due to version control.  Excel is also very familiar to John Q Public.  My generation has been using it since we were teenagers.  We don’t want to learn something entirely foreign and different.

You Don’t Have To Ask IT To Do Anything-  Many people would like a better solution including some basic workflow management ie- the equivalent of a damn dropdown to track things!  There’s one problem:  they have to go to IT.  That means the choice is now continue using Excel OR spec something out, sit down with IT, have them build/buy something that you probably won’t end up using anyway, test it, and then wait for it to be rolled out to everyone.  I’m pretty sure Excel wins a large majority of the time here.

Some Evidence of Excel Being Used as a Database

Public Relations- When we started Publictivity, it happened because a Public Relations firm was tracking all of their conversations inside of an excel spreadsheet.  They had a separate CRM system and would export the contact list to a spreadsheet.  From there, they would keep a spreadsheet for each “campaign” (specific product launch/set of pitches) with the status of the pitch, link to the coverage, and whether they received the review material.  If there were multiple people working on an account, they would just email the spreadsheet back and forth.  Crappy way to do things, but that’s what worked well.

Human Resources (UMiami Capstone Project)- For my senior year project as part of the CIS major we had to redo the system that recorded absences at the school of business for their employees.  My first question was: How do you do things now?  Their answer?  We keep track of it all in one spreadsheet that we’ve had for close to fifteen years.  That’s right, one spreadsheet, fifteen years.  I asked why?  Their response –  It was just the easiest way to do it, and asking IT to do this until now was a huge headache.  We eventually built a fairly simple and very cool system, but had to leave some things for IT to finish due to security purposes.  Since IT still has to do something that is > 30 minutes worth of work, I have a feeling that original spreadsheet may live to see another 15 years.

A Friend’s Startup-  The final inspiration for this post came about after sitting in on the pitch I mentioned above.  It was once again “another case of the spreadsheets”.  Right now many vendors track the risk to their supply chain in a spreadsheet in Excel.  For the first time since leaving Publictivity 18 months ago I realized that there is an intense amount of opportunity around displacing Excel as a lightweight spreadsheet + process management tool.  If large corporations are leaving risk management to an excel database, this has to be a very large problem.

YCombinator Idea #22- If you look at the YC ideas we’d like to fund thread, idea number 22 is rooted in solving this exact problem.

People often use Excel as a lightweight database. I suspect there’s an opportunity to create the program such users wish existed, and that there are new things you could do if it were web-based. Like make it easier to get data into it, through forms or scraping.

Don’t make it feel like a database. That frightens people. The question to ask is: how much can I let people do without defining structure? You want the database equivalent of a language that makes its easy to keep data in linked lists. (Which means you probably want to write it in one.)

The key here is the line that emphasizes the fear of databases to your average everyday worker.  The word database seems like a complicated thing that IT deals with, which might bite their heads off.  They’ve dealt with MS Access before, but that just scared them to death.

How We Can “Fix” This Problem

Be Wufoo’s cousin.  I’ve always respected Wufoo due to the simple, yet very painful problem they were solving.  This is explained well by pg in the ideas we’d like to fund post:

…In most companies the IT department is an expensive bottleneck. Getting them to make you a simple web form could take months. Enter Wufoo. Now if the marketing department wants to put a form on the web, they can do it themselves in 5 minutes…

Wufoo was able to take a process that might take months and reduced it down to a 5 minute process that anyone can do.  I think that’s the answer here at least in it’s simplest form.  Do what Wufoo did for collecting external data, but for collecting internal data and simple workflow processes. Take what is currently done in a spreadsheet as a database, and instead of requiring IT to take months to replace it with a custom built application, make a simple way to do it yourself in a drag&drop fashion in under 5 minutes.

Ironically without even knowing it, this is the problem I spent the last 6 months of 2008 solving at Publictivity.  We had originally built Publictivity as a way for PR professionals to better manage their workflow.  I realized we were just replacing a spreadsheet and that many other companies had this problem.  So we set out to build something that was malleable and any type of company could use to replace their “spreadsheet databases”.  It was a feature called Applets.  We finished it, but it was never released to the public, but here’s a demo video below (or direct link so you can watch in full screen HD).  Maybe I’ll put it online or open source it like I’ve always planned.

Some Notes:

  • Everything was completely drag and drop.
  • There were basic relational entities ie- you could reference the equivalent of other sheets.
  • There was a public applet directory where you could one click install templates other companies made publicly shared (you would NOT see their data of course).
  • We had basic data types that we felt every company would need (files, links, contacts, and actions).
  • You could easily set the header columns and set dropdown fields as filters.
  • All data was shared with your entire company.  We had started planning privacy/security features.  We also had a “Share” button that let you simply notify/share a team member of an entry.

There have been companies that tried to tackle this problem, but ultimately failed (Blist and Coghead).  It’s certainly a problem, but what is the right solution?

  • Pingback: James Fee GIS Blog » Blog Archive » This Just In: Excel Is The World’s Most Used “Database”

  • maacl

    Wasn't this what Dabble DB was?

  • maacl

    Wasn't this what DabbleDB was?

  • http://www.jasonlbaptiste.com Jason L. Baptiste

    It was in some ways, but I feel dabbledb was about large data sets, not really replacing excel as a database.

  • http://orestis.gr/ Orestis Markou

    This is what Resolver One of Resolver Systems tries to do. A spreadsheet that looks like a spreadsheet but does away with most of the maintenance horrors that Excel has (VBScript, mostly). It uses Python as the scripting language and can integrate with .NET libraries. Most advanced computer users can pick it up fairly easily… http://www.resolversystems.com

  • http://www.jasonlbaptiste.com Jason L. Baptiste

    What about novice computer users?

  • maacl

    Wasn't this what DabbleDB was?

  • Matt

    Excel isn't a database. The entire posting is ridiculous. Whomever wrote this obviously doesn't understand basic data structures. This drivel has no place among the venerable halls of internet discourse.

    • Martin Kuliza

      I Agree, Excel, By Definition is not a database. Excel is a spreadsheet, and a spreadsheet, by definition is (simply stated) “a calculating tool”.

      Excel has properties of the layout of a database, such as rows and columns, but the entire function of excel is completely different to a database.

      Using a spreadsheet as a database, is sort of like creating a template for an invoice using a spreadsheet as opposed to a Word Processor.

      I agree with the saying, if it’s not broke don’t fix it,
      but.. Please consider,

      If i used a hammer to hammer a nail into a piece of wood,
      that philosophy would apply.

      if i was to use the handle of a maglite torch to do the same job..

      we could argue that , it may work,
      but.. you can’t apply the same philosophy to this.

      the point being is.. if the user used the wrong tool for the job to begin with, that’ user error.

      Just like why people create invoices in Excel

      the thinking is…
      1. it’s easier to create boxes and borders where you want them on the grid, and that’s how it stars (and all because people don’t know how to create tables in Word,

      But then the trade off most people come across in Excel is
      if you have a cell and you enter a short invoice number
      and then 5 cells below it you need to enter a really long name (people to begin with get stuck on how to do this)

      if they started in a Word Processor
      they would not have had this problem

      the philosophy is..
      if it ain’t broke don’t fix it

      but there is also
      use the right tool for the job

      Everything has a purpose

      because Engineers design software, There is a specific purpose in mind
      and that software has been tested over time

      so.. Use it for what it’s intended to be used for.
      Simple

      I Agree that Excel has been used for many years in this way,

      But.. Then again People do actually use Optical Drives as Cup Holders
      and USB Ports to Power little fans on their Work Desks

      Finally,…
      Excel, used as a database has it’s limits.
      if a person has the intelligence to use excel

      THEN SWITCH TO MYSQL
      it’s just as simple
      it’s Free and much much much more flexible

  • http://www.jasonlbaptiste.com Jason L. Baptiste

    I'm not referring to using excel as an actual database for an
    application, but the fact that it is used by people all of the time to
    store information that should be in a db driven app. Whomever wrote
    this comment obviously doesn't understand how most of the world is
    working

    -jlb
    772.801.1058
    Discover Great Web Apps To Make Your Business More Productive At:
    Cloudomatic.com
    You Should Check Out http://www.JasonLBaptiste.com

  • http://blog.rosania.org Paul Rosania

    Wow. I certainly agree that Excel is a business process smell, in a lot of its use cases. In my few years as a consultant I saw it used for just about everything — CRM, document management, etc. However, I think one of the reasons it's been so successful in that role is because of how simple it is: just a stack of 2-dimensional sheets. MS Access can be used to build the types of apps that Excel is used awkwardly for, but most people can't grok the relational mindset. I wonder if there is a way to overcome this “flat” thinking that makes Excel so easy to understand for the average user.

  • http://www.jasonlbaptiste.com Jason L. Baptiste

    Hey Paul,

    Thanks for the comment! It's fairly nuts, but it just works for your
    average user. It's almost as if MSFT created the simplest product for what
    needed to be done, BUT it's not the right tool.

    I think people understand the sheets concept well. If you made it easy to
    start interlinking things ala cells on different sheets, it might work.

  • Jakewk

    Yeah, this was DabbleDB, which is now a take-under by Twitter, which means they'll likely shelve the product and use the talent on the team. Not an auspicious indicator for the robustness of the opportunity. However, it may be a function of web-oriented startup founders trying to address what is actually an Old IT problem. This may require a different culture than your typical Web2.0 startup.

  • Jakewk

    He's not saying Excel is a database, he's saying it's USED as a database. There is a difference.

  • Hang

    This article missed the most important point which is that Excel gives you a UI for free. I know a technologically savvy friend who deliberately chose to use Google Spreadsheets over MySQL for his startup for precisely this reason. Google Spreadsheets sucks as a database but it's great as a data manipulation tool. Same applies for Excel

  • http://www.ixxy.co.uk andybak

    You didn't read the article very closely, did you? You've missed the central point which is that people who might need a relational database (I assume that's what you mean by a database) use Excel instead because the alternatives are too difficult.

  • http://www.facebook.com/kuochiafu Kuo Chia-fu

    http://www.ragic.com
    I think they're doing exactly what you're talking about.

  • http://twitter.com/liveortell Chris Clark

    I think most of these issues are problems with users and processes, not Excel. IMO, a well-designed and thought out spreadsheet should be considered as a viable alternative to development in a lot of cases.

  • Rick – a Merican

    There may be no solution…it has been this way for a very long time…
    http://neopoleon.com/home/blogs/neo/archive/200
    and can be expected to continue.

  • Guest

    dfdfdf

  • Charlie

    Hey this Ragic looks really promising. Hope they'll make a difference.

  • Ken

    The person who claims this post is drivel because it claims that Excel is a database obviously can't read very well. (as pointed out by others) Also, using the strictest definition of a database, Notepad's txt format qualifies as a Db. (Stop the venum now, I am in no way claiming notepad is a good DB solution. I am just saying it meets the minimum requirements to be called a DB.) At one point, I didn't have access to a “real” DB, so I built a DB using SAS.

    Excel came in real handy with my web interfaces to flat reports and cusomizing calculations in the Excel spreadsheet. I'd be really hardpressed to do some of the things that were so easy that way using standard “DB” tools. I'm part of IT and I certainly can relate to problems working with IT people where more than 30 minutes worth of work are involved. I've wrtten specs that totally laid out the data design and 5 iterations and months later, the people responsible for providing the interface were still screwing it up.

  • http://qtp.blogspot.com/ sachxn

    yes the easiest things work for longer times….

  • Mouse

    I liked the Applets video. Concept is there, but man do I have some questions as to its use as a replacement for the numerous Excel projects out there. If it were ever to get to OpenSource, I would be test driving it.

  • http://stevenvore.myopenid.com/ Steven

    There may be no solution…it has been this way for a very long time…

    On my very first job (back before dinosaurs roamed the earth), as “the computer guy” for a decent-sized hospital's administration, the PC I was given had multiple low-end database packages installed (dBase II, dBase III+, Paradox and Reflex)… yet everything was done in Lotus 1-2-3 spreadsheets. When I moved their data into databases and produced reports showing what they asked for, they wanted to be able to make spreadsheet-like changes (not just formatting, but actually changing the data).

    It's not a technical problem, it's a user-need issue.

    • Steve

      dBaseII and III, even Paradox. I remember using all 3 very well. We really are before the dinosaurs. Didn’t know there that many of us left.

  • Bbalok

    I face the Excel versus a real data base system every day. Our company got to big and spread out geographically to use Excel and pass around a spread sheet for some of it's “applications” and that is when IT gets involved. One constant in these developed systems, I still have to provide a method to either dump it to Excel or import from a spread sheet.

    IT does take a long time to develop and deploy a system and seems to make it complicated from the users perspective. A lot of times it is complicated and IT wants to do the job correctly and takes the extra time to model a data base before development. The user just starts typing into cells and the data base evolves with the spread sheet.

    I have taken a new attitude and started to create some loosely structured data base systems that grow with the users needs. Not what I learned in school but at least they have an IT person involved. When it grows to be to big for the spread sheet I already have some structure and rules with the data, even if the users did not know it, and can now do the job for real.

  • Gigady Gigady

    Very interesting concept. I had an actual friend tell me that he actually looks forward for the half hour/day process it takes with excel duties though. It makes him feel more productive and in control of his information. He also mentioned that his IT's way of handling this process was effecient but he didn't like the fact that after the transfer he then now needed to go through IT for any little change. In the end it was still a process and he felt better by being in control of it himself.

  • http://www.jasonlbaptiste.com Jason L. Baptiste

    Ragic looks really cool. I'll play around more for an in-depth review.

    • jdwilson

      Guess you never did the review? I haven’t found one by a serious user anywhere.

  • http://www.jasonlbaptiste.com Jason L. Baptiste

    That's a really good point and one I should probably update the article with.

  • http://www.jasonlbaptiste.com Jason L. Baptiste

    Excel makes non geeks feel like empowered geeks, which is great. Excel is awesome at what it does here. My gripe is this: why haven't we made something better/evolutionary? Take the best parts of what Excel does as a database and build something people love. ie- fill in the gaps.

  • http://www.jasonlbaptiste.com Jason L. Baptiste

    Awesome. There's a very good chance it will go open source. Will drop you an email re: this. As long as it's a start, that's what matters :).

  • robertn

    Maybe the problem is not the spreadsheet but the storage and/or engine behind it. What if all the front side manipulations were still there for all the power users but the back side stored the data in a whole new way that would extend the capabilities more toward “databases”? And of course for sharing, the presentation of this spreadsheet would need to be through a browser ala Google Docs or Windows Live.

  • Richardc_2074

    So the real problem is not the use of Excel but that we've got to teach people to use cvs and get everyone to work with csv files.

    “Crappy way to do things, but that’s what worked well.” – a common theme of life in general really.

  • Phyroxis

    The old adage, if it ain't broke don't fix it, comes to mind.. What exactly is the problem here?

  • Pingback: EDGAR

  • Rico

    Tech savvy? If they were tech savvy, they could easily have built something for MySQL.

    Has nobody ever touched the largest desktop database out there? FileMaker is a great tool for this stuff.

  • http://www.astwoodconsulting.co.uk/ ms excel training

    Its rightly said
    that excel is the most used database.It has very functions and are very
    useful.I really liked your article.Thanks for sharing such a knowledgeable
    stuff.

    • LOL

      ^ LOL!

      • jdwilson

        Spam!

  • Anonymous

    Microsoft excel as
    a database for keeping track of information and basic businesses processes.
    This blog is very nice.I like it

  • http://www.simplyearplugs.co.uk jollygomez

    Great written article regarding excel.Indeed excel is very useful and most widely used database.I really liked your written skill.Thanks a lot for sharing.Keep sharing always.

  • http://www.Tramadol-Addiction.net Tramadol Addiction

    excel are work everytime as a database.

  • http://www.creativementor.com.au/microsoft-excel-training.html Excel training

    asasasas

  • http://www.meritline.com/iphone-stylus-touch-pen---c-14709.aspx iPad Stylus

    worlds top digital goods are here. i like it.

  • http://www.meritline.com/iphone-stylus-touch-pen---c-14709.aspx iPad Stylus

    worlds top digital goods are here. i like it.

  • Terri Lawton

    Great post. Here’s a post that shows you how to convert your excel to web in minutes http://blog.caspio.com/integration/convert-ms-excel-to-web/

  • http://www.run8tech.com/ Nate

    I think having spreadsheets that have VBA coding in them to extend the functionality may be a partial answer.  At least for Revision Control.

    http://www.run8tech.com/tools.aspx 

  • http://www.facebook.com/profile.php?id=827888417 Charles Var

    TrackVia.com 

  • http://www.fluto.co.uk/ dazy

    Most blogs you read online are boring and don’t offer much information. This was really helpful.Its really a superb post.

  • http://twitter.com/BeoMartin Martin

    Sorry for warming up this old Blog, but you missed some serious issues:
    1. scalability, if you have 20 users working on one spreadsheet, even if they knew about subversioning, it would be horror to merge all changes.
    2. excel has limitations, especially excel 2003. Limited line numbers and cells
    3. imagine the whole corporate data of a company is stored in 1,2 or 3 excelfiles or even in 20. What prevents a user of dropping one or all of these files to the public? It’s far to easy to move the data to another location. And you can’t even tell, who leaked it.

  • ExcelVBAJunkie

    So you made a little bs database. So what. It is not “infinitely useful”. It sucks! Excel-VBA is used because it is the best analyzing tool ever made. It can be a form, query, function, report, simulator, video game, statistical engine, etc. What ever you want it to be. It is so flexible its ridiclous. Yes some use it to store simple information and you could structure that into a database if you want but who gives a shit. If you need a database then just store the data in Access. But use excel when possible for your front end and reports sending and pulling from Access seamlessly. If you know vba you can quickly and easily talk to any Microsoft Office product. I make Office one nice sweet program using VBA. You should try it and stop listening to the comp sci idiots! Go talk to any senior quantitative analyst in any company and see what they use. 9 out of 10 times its excel

  • ExcelVBAJunkie

    Hey Cool! Your read my message. I understand your reason for deletion. But I am not wrong. From a programming standpoint its neat what you have done. But for anything complicated and flexible it is not gonna work. Have you ever made super sophisticated calculation engines? Monte Carlo and advanced stats to find optimums? Quickly, Easily and with complete flexibility. That is where Excel-VBA rocks. Yes it’s not ideal for a multi-user database. But a single user at a time it’s fine. So don’t use it as a multi-user database. Use Access in that way but don’t lose the power of Excel. Send code seamlessly through the two and Outlook, Word, PowerPoint. So many people have been taught to be elitist about Excel because they learned some random programming language. I destroy anything they can do in simple old Excel VBA.

  • Scott Carter

    Great post. Another related and more informative blog on creating a database from excel http://blog.caspio.com/tech_tips/how-to-create-a-database-from-exce-import-spreadsheet/

  • sross

    Another alternative is IBM Cognos TM1.
    It offers a real time approach across multiple users without the need to email files around. It is not a relational database, its OLAP. It is easy to use, can manage large volumes of data, has an in-built calculation engine and even a web interface. You can also slice from the database to an excel spreadsheet. I used to love Excel, with vba, however now I work with TM1 as a consultant in the UK, and I will never go back to Excel! steveross_01@hotmail.com

  • kevinmorice

    “At this point, the headaches usually start rolling in due to version control. ” – Review, Share Workbook. – Done.
    PR example – Add Auto filters. – Done

    The reason we use Excel for large data sets is not just familiarity, or ease of use, or cost, or portability, or even the ability for a trained monkey to do quite advanced data manipulation. It is a combination of all of those.

  • attila

    I’ve just started a blog about the options to choose from when someone decides to move an Excel spreadsheet to database. You can find it here: http://excel-intelligence.com/