The project is called QueryStorm. It uses Roslyn to offer C# (and VB.NET) support in Excel, as an alternative to VBA. I've posted about it before, but a lot has changed since then so figured I'd share an update.
The current version includes a host of new features, namely a C# debugger, support for NuGet packages, and the ability to publish Excel extensions to an "AppStore" (which is essentially a NuGet repository). The AppStore can be used by anyone with the (free) runtime component.
Another great addition is the community license, which is a free license for individuals and small companies to use. It unlocks most features, but it isn't intended for companies with more than 5 employees or over $1M in annual revenue.
I would love to hear your feedback and am happy to answer any technical questions about how QueryStorm is implemented.
This is excellent. I can see buying it for my business. We already use Excel extensively for the open file format; that it's local first; that it's just a file; the low ceremony of getting started; the multipurpose grid for code, data, and presentation; the ubiquity.
The more low-code/no-code SaaS products I've tried, the more I value it. LC/NC SaaS generally is expensive and inflexible. Everybody knows Excel, it's dirt cheap, and it just works. It's part of a certain IT subculture to bash it, but I stopped laughing along.
Excel is great at what it does. Love seeing clean solutions like this that give it even more muscle.
Oh a plug-in, ha, for some reason I read the title and thought this was a 'just to show I can' type implementation of all those things actually in Excel, 'Doom in Excel' sort of thing.
This seems far more useful!
I also thought this was going to be an article about implementing a C# compiler in excel! This on the other hand seems like a very neat product.
Thanks! Yeah, I probably should have made the title "...in Excel" instead of "...inside Excel". I wonder if more or fewer people would have clicked on it that way.
Yeah, I thought you crafted some Excel monstrosity based on the title. Maybe a mod can change the title for you to say "for Excel."
Also, I would be super proud if I'd created this. Excellent work!!!
If we're being pedantic maybe "for Excel" would be clearer? But what do I know.
"working inside"?
That would be my choice too. Both "inside" and "in" imply "in the spreadsheet". "For", on the other hand I would understand as "for the Excel program".
I had the same thought lol. Honestly, the fact that such a thing seems possible at all should be props to the people who built Excel
This is incredible. I work in industrial automation and I often have to manage / make edits to "point assignment charts" which are essentially Excel Workbooks with 100+ Worksheets, each containing ~10-100 rows of formatted rows which define various parameters each real world "point" should have in order to communicate with RTUs / various SCADA nodes.
Sometimes the feedback involves making a minor tweak on every Worksheet. Sure you can muck around with VBA (which is a horrible language and experience), but I often just do it manually to get it over with instead (click click click... very tedious).
C# is my go-to "business logic" language as a developer, and this looks so well integrated, really excited to use it.
You have inherited a crappy database. Export it via ODBC and use more powerful tools.
Would love to know how you get on with it. If you'd care to share, please to let me know.
> "Sometimes the feedback involves making a minor tweak on every Worksheet."
Could be a good task for PowerShell and the ImportExcel module[1].
(PowerShell also being a .NET language and written in C#; it has convenient access to COM control of Excel if you can't import the spreadsheet and export an updated version; through $xl = New-Object -ComObject Excel.Application )
This is absolutely awesome.
The combination of a spreadsheet, database and data scripting runtime would be extremely powerful.
This is where Airtable, nocodb and baserow are candidates.
I worked at a company that used gridgain with Excel as a cluster to process models on lots of servers.
I suspect there's a lot of low level Windows integration details about how to integrate with Excel. COM objects and DLLs.
Why is integrating with software so difficult? Very few people know how to do it.
How you design software that is extensible?
Atom was extensible and VS Code is less so.
Why has nobody integrated the web browser with QT? So we can browse into a Qt application. So you can create desktop widgets and canvases with Qt from Javascript? That way you could ship desktop software and arbitrary integrations to software.
> Why is integrating with software so difficult? Very few people know how to do it.
I feel it's because of inherent conflict of interests and priorities. A software product can provide services to the user, or provide services to other programs. It's rare to see one that does both, because the two goals seem often at odds.
Say you're writing a software component for other programs to use. You probably have some specific programs in mind, some of which are end-user-facing. In this case, you may not want to spend resources on developing user-facing UI and features that make your product usable stand-alone - you'd likely end up competing with your own customers.
Say you're writing user-facing software instead. You want to own the entire user experience. Mixing UI, user interaction and the "business problem"-solving code together lets you work faster. Also, you don't want to make it easy for anyone to integrate with the core of your software, because that would be inviting competitors to own the user experience while using your own backend, ruining all kinds of marketing shenanigans you'd rather play on your users.
I don't like this, but I see there's a mix of both reasonable and greedy reasons why software isn't developed with integration in mind.
The major exception here is when the platform on which the software runs encourages, insists on, or demands the software to allow for integration. Windows does that to an extent with COM / DCOM. Emacs does that by its nature. But it's only moving the problem one level up - e.g. Microsoft environment can't integrate well with Apple environment.
OT: You can do that with Qt. We played with implementing it ourselves for a project about 10 years ago. Since then I think there is an official QPA that will render Qt apps within the browser, and there is the Wt project. Might be worth looking at for some needs but I don't see it becoming popular.
Really cool. I could see MS jumping on this because of their strong association with C#.
That said, I think the ideal replacement for VB & VBA in most contexts is python. The reason is they're both high level languages with ease of learning as a selling point. Python has a massive userbase, and there is significant overlap of python and MS Office tools. Data analysts who have to deal with Excel & SQL Server are much more likely to know python than any other language outside of SQL.
It seems like python has pretty much locked up the spaces that VB was designed for.
I thought I remember hearing like 4 or 5 years ago that MS was planning on replacing VBA with Python Support in Office apps. But I can't find anything to back that up, so maybe I just had a really wishful dream.
They might have some involvement in python libraries or tooling to interact with excel files from python apps, but the default programming support within excel is still VBA. They did add a javascript API, but getting that up and running is a job geared more for a dev than a data analyst.
What I think they should do is not just make a python API, but replace VBA with it, and expose it to end users where VBA is currently exposed. Python should be the language of macros and user defined functions.
They should dust off Windows Scripting Host and use that as the nexus for Office app scripting.
Looks awesome. One fundamental issue I always faced with Excel calculations of any kind (formulas or vba) is the lack of ability to source control cleanly (binary format plus data and code are intermingled). This weakness is the other side of the spreadsheet’s greatest strength which is low friction experimentation with data. Does this product attempt to do anything here?
Thanks!
You can version control the code you write with QueryStorm, to an extent. There are basically two kinds of apps you can build with this: extension apps (where you build general excel functionality) and workbook apps (where you automate a particular workbook).
The code for extension apps is stored in a folder on your machine, and you can version control that easily.
The code for workbook apps is stored inside workbooks. While there's no version control functionality in QueryStorm, it does let you export code from a workbook into a folder, as well as import code from a folder into a workbook project. There are context menu commands for this in the code explorer pane. This lets you decouple the code from the workbook.
Basically, for version controlling code in a workbook, you'd have to export the code into a folder and version control it there. Exporting into a folder will clear everything from the folder except hidden stuff like the ".git" folder so your git repository will be safe. Not a perfect solution, but it can be done.
Have you considered implementing a project folder structure, so that the workbook and the source code could be version controlled together in git, but the IDE would import and export the source automatically from/to the workbook?
You mean add the workbook itself into the folder as binary file to include in source control? I haven't, but it might make sense, to keep the workbook in sync with the source code.
Since XLSX etc. use Microsoft's Open Packaging Convention (a ZIP container consisting of mostly plain text), consider adding native support for unpacking and normalizing the payloads so they can be trivially diffed by existing source control systems.
It's something to consider, for sure. Perhaps there's an easy win to be had there.
Can you also import it from the folder?
Yeah, you can import the folder back into the project and it will replace all of the existing files with the ones from the folder.
Looks great, but I ran into an issue I'm hoping you can help with. I copy/pasted the very first example from the blog and it doesn't compile. I also have been unable to find the documentation necessary to get past the problem.
It's the example that reads the files in a dir and makes an Excel table, and it fails on line 16, which reads `(excel.Selection as Range).WriteTable(files, "myNewTable");`.
The error is `Argument 2: cannot convert from 'System.Collections.Generic.IEnumerable<<anonymous type: string File, long Size>>' to 'QueryStorm.Data.ITabular' | `
I can't find docs for WriteTable() [even after finding those gitlab repos that contain QueryStorm docs].
I think anyone trying to run with this, will need access to docs on the API.
Thanks for trying it and replying back here.
It looks like I was using a pre-release version of QueryStorm while I was writing the blog post. I just released that version so if you restart Excel you should get an "Update available" button in the QueryStorm ribbon in Excel.
Could you update and post back if that fixed the problem?
Another way of "fixing" it would be to replace the following line:
(excel.Selection as Range).WriteTable(files, "myNewTable");
Write(files, "myNewTable");
The documentation does lack detail in some areas, which I basically cover by answering questions via email. I do need to invest time every so often to update and extend it.
I work on Excel at Microsoft and this is really cool. I’ll check it out and share it with the rest of the R&D.
Do you know if this item from the Microsoft 365 roadmap would stop Add-ins (xll files) like this from working?
https://www.microsoft.com/en-us/microsoft-365/roadmap?filter...
I also have a vested interest as one of my software products is an xll Add-in. Any further details from Microsoft would be most welcome, thanks.
No idea, haven't seen this until now. Hopefully they provide a mechanism for allowing legitimate addins to get around it. Perhaps using a code signing cert (which I do) or an EV certificate would be appropriate. Security has been a bit of nightmare with Excel addins. I've had to deal with false positives on and off again a bunch of times already. First it was windows smart screen, then antivirus vendors every so often. It's a slog...
Typically you can go into the file properties and check the “Unblock” checkbox to remove what Microsoft calls the “Mark of the Web”.
Open us a feedback through the app. We actually read every tiny bit.
Cool. How can I turn off the animated cursor movement in Excel? And Outlook? Drives me crazy. I keep submitting feedback. Even a registry hack is fine.
Thanks, that's great!! <3
Please tell Satya to show the ancient C SDK a little love. I'd hate to see this disappear: https://github.com/xlladdins/xll#fp-data-type
Jan 2024 - Microsoft releases Excel with it's own built-in C# IDE, doesn't credit original developer.
Oh boy you don’t know what our culture looks like. It’s more “it’s cool that people are building on top of it and we want to figure out how we can make it easier”
People didn't forget this story: https://www.theverge.com/2020/5/28/21272964/microsoft-winget...
I always got the feeling that because it's such a key product the Excel team was basically its own organisation within the wider corporation and as a result had a lot more freedom to make its own choices.
(not that I'm arguing for trust that would, as you say, be misplaced, but if my feeling is correct then the Excel org's englightened self interest can diverge from the mothership's self interest more than other divisions can, which makes the calculations as to what your management is likely to decide notably different)
I got this impression as well. I worked on a team two years back building out a very complex Excel plugin using Excel.js. The SDK has quite a few quirks and we ran into some problems along the way, but the team at Microsoft responsible for it was happy to help and interested in hearing feedback and features we wanted. A positive experience :)
They might be likely to offer him a job there honestly. The architect of C# was poached from Borland after all.
I hope it does work out for the original author and he deserves it but there have been cases before where concepts have been taken from their originator and implemented by the large corporate with no cash or even credit given.
Maybe this was a better initial comment.
No he wasn't, he left Borland after being disappointed how things were going.
He declined several approaches from ex-Borland people working at Microsoft, before reaching that point.
https://behindthetech.libsynpro.com/001-anders-hejlsberg-a-c...
How'd that go for the AppGet dev?
I always felt that there was more to that story than what was reported. My understanding is they reached out to him to interview for a PM role to develop an official "AppGet" at Microsoft but, for whatever reason, he didn't get the job. Maybe the interviews didn't go well? Maybe there were some red flags somewhere? Who knows. But it didn't really sound to me like they acted in bad faith.
Nor was I. It's worth a search and read.
Even today, it isn't Embrace, Extend, Employ.
Honestly it's annoying that they haven't already done that. I was surprised a few years ago when I checked and it didn't exist.
Looks great, will give it a try (as a fellow Excel developer).
One small point: on your webpage the "Try QueryStorm out" button on the "Start a free trial" panel just points to: https://querystorm.com/csharp-in-excel/ rather than the generate key page.
Thanks, good catch!
Hats off @anakic this is class! It would be a game-changer if Microsoft licensed QueryStorm built in to Excel. Let's hope the right people at Microsoft are paying attention to you.
Thanks! Wouldn't mind that outcome one bit. It's hard to get exposure and adoption for a platform like this without some serious muscle backing it. I suspect Microsoft has its sights set on the cloud, though, but still, it would be amazing if they adopted QueryStorm.
Wow. As someone that has written a fair amount of VBA in the past, this is a game changer.
Impressive feat of integration and I hope you're rewarded for all your efforts.
Clearly a passion project!
Hey Anakic, I remember posting a link to ThingieQuery here many years ago (2016 maybe?). Glad to see it's gone from strength to strength!
Yeah, ThingieQuerie it was called back then:) Thanks, I appreciate the kind words!
Microsoft actually built that already, then they decided to kill it. It was called VSTA (not to be confused with VSTO). It shipped with one of the apps of Office 2007.
At first I thought you were referring to VSTO, then I saw the "not to be confused with VSTO" part. I had no idea this ever existed. Thanks for mentioning it. I wonder why they decided to pull the plug back then.
I suspect it must have been part of "let's try to secure / lockdown office", "users writing codes is bad, they should only push buttons", and at the same time they tried to make it as hard as possible to use VBA (all sorts of warnings and things disabled by default, etc)
I am also aware of a commercial product that offers a .net IDE to be embeded into your software, though I never tried it: https://www.alternetsoft.com/products/scripter
Funnily, QueryStorm uses VSTO behind the scenes: the IDE component is a VSTO addin.
This looks great! In my previous company, we somehow ended up building a whole ERP around Google Sheets. Google Sheets is a very powerful tool, but the JavaScript definitely felt clunky and outdated.
A solid C# foundation for building on top of excel could probably be very useful for some companies.
I do wonder though how you can nail down a target audience for this kind of tool, seems like you'd need a special kind of tinkerer and I'm not sure how many like that are out there. In my other company, which was a small company, the ones who headed it were essentially engineers that transitioned to executive positions. I doubt there are many like that out there.
Very cool. Should be a part of Excel tbh. LINQ is much better suited for data munging than JavaScript.
F# is even better.
Maybe, but I think C# has the advantage that "everyone" knows it or Java so it is trivial to get started.
Last I checked, admittedly a couple of years ago, starting with F# wasn't absolutely trivial.
It is, if you know where to look.
Therein lies the problem. The language doesn't have much marketshare, and few champions.
I just finished reading Scott Wlaschin's Domain Driven Design for Functional Programmers and it's an incredible, batteries-included guide that goes from "hello world" to "and here's how you use it in an organization" in a few hundred pages.
I wish I had been exposed to it about a decade ago. Incredibly easy to follow and well-explained.
I think that the problem is that:
- fsharp's big selling point is that it runs on dotnet - most FP ideologues run screaming from M$, and there are other, better (subjectively), more-active FP langs that do many of the same things, such as OCaml and Haskell - most dotnet shops are "csharp or die" and getting fsharp adoption is more or less an impossible task for cultural reasons
I support fsharp and generally like it, but I'm not reaching for it for net-new stuff that doesn't already have a dependency on dotnet. $0.02.
What’s the markershare though? F# is great but Microsoft made it sortof a second class citizen
For that you already have PowerQuery on Excel.
As for being better, F# is better on the language level, but sadly sucks on the tooling effort for .NET frameworks, that Microsoft is willing to spend on it.
I tried clicking on the animation to make it bigger, but the larger image doesn't load.
I'm using the Brave browser.
Same on FireFox.
Yeah, I just checked and it fails on some gifs for me as well (but works on others for some reason). I'll leave it as is for now, best not fiddle with it while this post is trending.
This is such an impressive feat of engineering. Excellent job. I have no doubt this will be a hit. It wouldn't surprise me if MSFT just decided to make you an offer for it.
Would this work with Excel for the Mac as well or is primarily a Windows thing? Somehow, while good, I feel like Excel for Mac just isn't as good as Excel for Windows.
Unfortunately, no. Excel for Mac doesn't support either of the two technologies QueryStorm uses to talk to Excel (namely VSTO and ExcelDNA).
Might be possible to build something that would work across all plaforms using Blazor and the Monaco editor, but that would be another huge project.
No questions, just complimenting on what looks like a great, very very useful product.
Many thanks!
Looks awesome, and my favorite thing is this:
> $379 License + 1yr support
Cheers to you for not collecting a recurring fee for a desktop app!
The license is for 1 year of latest versions of the application, after that you'll stop receiving updates. This is pretty common for desktop applications, compared to SaaS.
It's becoming less and less common, even for desktop applications - which is why I thought this warranted some congratulations
I remember you back when it was just an SQL plug-in for Excel :)
Congrats on building this and sticking with it so long. It’s awesome!
I would have used the hell out of it 8 years ago! VBA IDE was a pain.
Great work!
I clicked on resellers & affiliates because I was curious about your terms. They are both empty.
Yeah, didn't get around to populating those yet. One-man show, so things slip by...
ComponentSource is currently the only reseller: https://www.componentsource.com/product/querystorm-plugin
Thanks for the heads up!
Let me know if you want to chat about sales/marketing linkedin.com/in/ckluis
Good work, but this is depressing:
"VBA is a bit (a lot) behind the times and Microsoft isn’t planning on upgrading it. Instead, they’re offering a sandboxed JavaScript environment called Office Scripts."
Microsoft - Just swap VBA for Python ffs.
Find a way to connect this to Zapier and the Universe might open up a wormhole.
Now I can use excel. I always sucked doing stuff in excell formulas. With this now I can just linq and spit the info I need.
Now sure how known this is, but Excel 365 for Business / Enterprise supports Office Scripts (under a "Automate" tab), where you can create scripts in TypeScript.
https://learn.microsoft.com/en-us/office/dev/scripts/overvie...
https://learn.microsoft.com/en-us/office/dev/scripts/develop...
devs "hate" excel because you have to reference by col/row (C10 = tax rate, etc). But few devs know that you can name cells (and ranges and tables). Just to the left of the formula bar you'll see a textbox with the default name of the cell (C10). Just delete C10 and call it "tax_rate" and now you can reference "tax_rate" in all your formulas.
Don't show it to my boss, please.
It is super cool and would love to play around with it - just not at work :)
Impressive, Excel would benefit having this built in
Animations embedded within text are distracting and make the reading experience worse
I like that a gifs shows so much in very little space, but I take your point about it being distracting. Regular mp4 videos you can play/pause explicitly might be a better option for the future.
Does this use https://excel-dna.net/ behind the scenes?
Yeah, the runtime component is an ExcelDNA addin. The IDE component is a VSTO addin. Since VSTO doesn't allow defining custom functions I switched to using ExcelDNA for the runtime.
What did you use to write the installer, and how do you determine if the targeted version of Excel is 32 or 64 bit?
I've built an Excel-DNA based Add-in (https://www.excelpricefeed.com/) and distribute it via an installer built using Advanced Installer. It works well, the only real pain point is the the user has to find out which version of Excel they have installed in order to choose the correct download (32 or 64 bit).
I use Wix for the installer. Can't remember how I dealt with that particular issue, but ping me at antonio [at] querystorm.com in a day or two, if you're interested, and I'll have a look at how exactly I dealt with it.
Please consider acknowledging that on your website.
I would love to but I'm a bit scared to fiddle with the post now while it's trending. I think it's mentioned in a few places on the website, but not in this blog post. I do also use a bunch of other libraries, but yeah, ExcelDNA is one of the critical ones so mentioning it would be in order.
Linq in Excel is very interesting.
Love it! Big fan of developer productivity tool like Linqpad.
Will this only work on Excel running on Windows 10/11 X64?
It works on Excel 2013 onwards but only on Windows. Both x86 and x64 are supported. Mac isn't supported I'm afraid.
Show stopper for me, but can I just say what a fantastic product and the article introducing it kept me hooked to the end!
I have to do my own marketing, so I'm trying to blog more often these days. Really glad to hear you enjoyed the writing!
Is there a technical blocker on macOS Excel side that prevents you from implementing it, or is it just a matter of priorities?
A bit of both. On the Windows side I can use VSTO or ExcelDNA to integrate with Excel, but neither is available on the Mac. One alternative I'd like to try is a web-based version which would use Blazor and the Monaco editor but I'm really not sure if that would be at all doable. I don't have the time to go there at the moment, but I'd love to try at some point.
This is pretty mad.
This is excellent. Great work!
Installed :)
Beautiful.
So can people upload their code snips to the appstore such that, say, a marketing data analytics person with no coding experience can pay to get 'buttons' to run queries that are relevant to their needs?
like the salaries example, for example, I used to spend a shit-ton of time educating a CFO on AWS spends in excel.
If I could have a code snip 'button' to use some AWS-CLI creds to pull results directly from AWS asa "CFO BUTTON" and set them up with an easy export to an XLS - and then let them build their own dash to their choice that would be cool.
I havent looked at Cloudability in quite a while, (know the founder) -- but giving a CFO-type an excel button... Every CFO can navigate excel. so they get an invisible tool that empowers them, and relieves stress on the Ops team...
Yep, that's the idea.
You create a project that defines some custom functions, ribbon commands, context menus, shortcuts etc... and then publish it to a store. Then anyone who has the QueryStorm Runtime (and the url of the store you published to) can install it through the "Extensions" dialog.
I haven't yet set up the ability to charge for packages but I plan on offering this. I first need a community of users, so didn't focus on charging yet.
If you'd like some help setting up something like that AWS scenario, reach me via email: antonio [at] querystorm.com, I've done several such integrations already (Google analytics, Monday.com, TSheets, Google drive).
this is.... awesome.
[dead]
[dead]
If the user can code in C#, why they need Excel? Exporting to Excel for other non-coder employees, sure why not.
I don't understand the use-case, Excel is horrible tool for anything else than stupid, a few hundred rows data science.
You must not work with any kind of real world data if you think Excel is a horrible tool.
Pulling in 1000000000 rows of data you know absolutely nothing about to throw into your ML model and have it spit out some questionable result doesn't mean you actually work with real world data.
>a few hundred rows data science.
People that work with a few hundred rows are called "almost every office employee ever" and not data scientists. And you are naïve as hell if don't realize how powerful excel is for every random ass thing that happens in the real world.
Hope you give it a go. I'd love to know how you get on with it if you do!
I love how approachable it is. Domain experts in other fields are able to program in excel and codify their knowledge. While, not always great in terms of code quality, often it is bad, it still provides a great starting point in moving to something more resilient and safer in terms of data validation.
There is a reason it is the worlds most poplar programming language
"most poplar programming language"
In your opineion.