Happy 1st birthday, Deneb!
As I sit here, it’s incredible to think that a year has gone by already since Deneb became generally available (GA) in AppSource. It took a little while to get there though, and for anyone not familiar with Deneb’s history, here’s a quick overview:
Both languages are from the work at the UW Interactive Data Lab, and its many alumni and contributors who continue to evolve it: led by Kanit “Ham” Wongsuphasawat, Dominik Moritz, Arvind Satyanarayan and Jeffrey Heer, but be sure to check the contributors (V | V-L) to see just how many folks have helped get the languages to where they are today. And, without such great and versatile languages, we wouldn’t have Deneb!
It’s been about two and a half years since I had the notion of making a visual that could parse Vega specifications in Power BI…
About a year and a half ago I’d spent enough time on it to make it available for public preview.
And, after some further iteration, it became available in AppSource (and certified) in November last year.
From inception through to GA and beyond, I’ve been keeping track of some of the things that have happened and thought it might be good to recap what’s taken place (that I know of).
Community: from preview to GA
Power BI’s community is awesome, and one I’m really fortunate to be a part of. With Deneb, the Power BI community did their thing: got curious about it, explored the Vega languages and then started to help others learn how.
It’s probably also worth mentioning here that because Vega and Vega-Lite are not proprietary to Power BI, there’s also many places outside of the Power BI community where you can learn from others. You can refer to Deneb’s support page for details of some of the popular communities where you can network and ask for help.
I’m really stoked that Deneb picked up some really strong supporters at the idea stage and helped convince me that it could potentially work enough to finish-off prototyping it and putting it out there. We also gathered some more enthusiasts during the public preview phase and these folks both helped to promote the possibilities through creating and sharing their own content, and became a great source of feedback for moving Deneb towards GA.
Special thanks here go to Kerry Kolosko, José Escalante and Greg Philps for their early support and great content, and doing the hard yards to help make the languages easier for others to approach in Power BI so early on. Thanks also to Reid Havens and Mike Carlo for inviting me onto their YT channels and presenting to their audiences. Mike has also started and curated a GitHub repository of Deneb templates that others can contribute to, and this continues to grow. We also got some support during the early stages from Power BI user groups, including:
- Microsoft Power BI UK - thanks to Leon
- Excel and Power BI User Group, Wellington NZ - thanks to Phil & Jeff (Jeff particularly for putting a roof over my head for the night)
- Queensland Power BI User Group - thanks to Craig and Gilbert
- Austin Power BI User Group - thanks Matt, Mike, and Sean
- Canberra Microsoft Analytics & Power BI User Group - thanks to Jim, Angela, and Kieran
Community: from GA to now
Since going to GA, we’ve seen more people in the community pick-up Deneb and create additional content. There’s been enough traction where I’m now struggling to keep up with what people are doing (which is kind of awesome in and of itself), so here’s some of the ones that stick out:
More user groups:
Thys van der Westhuizen has been creating a heap of content, including reproducing the Big Book of Dashboards chapter-by-chapter. Thys’s site also has loads of posts showcasing and explaining further ideas and content.
At the recent New Stars of Data, Blanca de Erausquin’s speaking debut was on Deneb (and it was a fantastic session).
Enterprise DNA has got behind Deneb in a big way:
- There’s a forum dedicated to Deneb that contains all of the templates that Greg Philps has been demonstrating.
- They’ve hosted Deneb-based sessions at their community events, including those from Kerry, Greg and myself.
- Greg created an introductory course, which is a great way to see the potential of the languages and how they can be used in Power BI.
- There’s also now a downloadable cheat sheet for common scenarios (also by Greg).
To everyone (especially those I may have missed), thank-you for sharing your learning and experiences, promoting, and presenting about Deneb and the Vega languages, and for helping others to learn and grow. You’re all truly awesome :)
And, if anyone wants to let me know of anyone who I have missed, or if you want to get their (or your) content added to Deneb’s Resources page, please feel free to let me know. I’d love to host it there if you’d like to share it there.
My experiences of the journey so far (development-wise)
I’ve talked briefly in various places about some of the rationale and technical challenges around Deneb, but I don’t think I’ve spoken cohesively about it and as such I thought I’d dedicate some time to this, as it may be interesting to some.
For anyone who has tried to do it, building Power BI visuals using the SDK requires web development knowledge, and while anyone can do this (myself included), it’s a very steep curve if you want to explore a visual idea or have to learn on demand. It’s also a big step taking a visual idea that works well with a static datset and viewport size, and making it much more dynamic, in the Power BI sense of things.
Ever since I found out about the Vega languages, I thought they’d be a great fit for those of us who like to build our visuals in a more declarative manner, and the idea of a runtime environment in Power BI that houses the languages (avoiding the need to build and test them separately) looked like an ideal middle-ground from my perspective. As Vega is quite extensible, there was potential to integrate some of the interactivity features that are expected from Power BI visuals without running into some of the immediate limitations when using R and Python in Power BI.
Making (what eventually turned into) Deneb turned out to be a lot more complicated than I thought it might be. It’s certainly been a labour of love rather than an entrepreneurial endeavour, and completely researched, developed, and supported during my free time (and for free). As such, there will always be concessions that need to be made to get something out there. If I would have waited until I was happy with it, I’d probably still be writing that first version. And I’m glad I did put it out there, as I’ve already learned so much from those who have been using it.
When I started, I thought that the ideal path would be to leverage work that the existing Vega community had done: Vega Editor is a great tool and it would have been better if I could have just lifted and shifted this code into Power BI. It would have made sense - if possible - to be able to re-use (and ideally contribute to) these tools rather than re-inventing the wheel.
I definitely tried to do this first; however, the realities of making this work within a custom visual became apparent quite quickly. Wanting to ensure that the intended solution could work as a certified visual further compounded this (therefore creating more limitations before I began).
Vega-Editor has pretty much everything I think you need for authoring specifications, particularly as it uses Monaco Editor, which is Microsoft’s editor component behind Visual Studio Code. This has all kinds of neat features - like autocompletion from Vega’s JSON schemas - that are great for productivity.
After a lot of trying to make it work, I deemed it not possible - at least within the constraints of my knowledge at the time - so had to abandon that idea. I’ve tried many times since, including getting help internally from MS on how I could approach it, but it just isn’t possible to make it work easily within a custom visual without having to load all its dependencies from a remote endpoint (and that’s a big no-no for any certified visual).
Because of this, I decided that I would need to focus on creating my own personal interpretation of Vega-Editor, working to these limits.
Looking back at how much I dedicated to researching and developing Deneb over the last 2 years or so, I can safely say it’s been well over 2,000 hours of my personal time to make it happen. My wife has been incredibly understanding and supportive, and I’m extremely grateful to her to have been able to have the opportunity to dedicate to figuring out how the idea could be developed. I mean… I’m still figuring that bit out, and I’m really happy to see that its genuinely helping people do more inside Power BI. I’ve had some great feedback from people all over the world, and I hope that many more discover the Vega languages and can see how they can be applied to Power BI as well.
Deneb has been incredibly useful in my work and for all challenges I’ve had (so far), the Vega languages have been up to the task. I’ve been fortunate to have been able to use Deneb visuals in a number of projects at DiscoverEI, as well as helping other folks (from a professional services perspective) to build their own for a fraction of the cost and effort of starting from scratch.
Through learning and using I have my own list of things I’d like to see, and in addition to this, folks have added their own ideas to Deneb’s GitHub repo. I’m now trying to learn how to wear a product manager’s hat in addition to the development and documentation work; there have been a few big updates since GA, and we’re continuing to move forward.
With each of these new releases, I’ve also been putting a lot of time into refactoring code and endeavouring to find a better editor component that can work in a custom visual, but it just hadn’t been possible to get there yet. I was also finding a lot of the newer features I was looking to do in version 1.5 and beyond were made more difficult by the technical debt that had accrued over Deneb’s (short) lifetime.
I talked a little bit about these challenges in the 1.3 release post, and how I was planning to move forward. This stance hasn’t really changed, but despite saying that 1.3 would be the last significant update, there was enough work to warrant a 1.4 release. I just can’t stop tweaking this thing…
I was hoping to dedicate some space to start talking about where version 2 is currently at, and what it might look like, but this post has run a little long. I promise I will share some details in a follow-up post, but this might be in a couple of weeks, as I’m currently preparing for Difinity, which will keep me busy for the rest of this week and most of next week. Bet you can’t guess what I’m presenting on…
So, for now, I want to say thanks again so, so much to everyone who has found Deneb useful for their needs. I hope that you continue to create great things and will look forward to what’s planned.
As always, thanks for reading. I wish you all the best, and stay safe,