Daniel Marsh-Patrick
Daniel Marsh-Patrick

Happy 1st birthday, Deneb!

Happy 1st birthday, Deneb!

As I sit here, it’s incredible to think that a year has gone by already since Deneb became generally available (GA) in AppSource. It took a little while to get there though, and for anyone not familiar with Deneb’s history, here’s a quick overview:

From inception through to GA and beyond, I’ve been keeping track of some of the things that have happened and thought it might be good to recap what’s taken place (that I know of).

Community: from preview to GA

Power BI’s community is awesome, and one I’m really fortunate to be a part of. With Deneb, the Power BI community did their thing: got curious about it, explored the Vega languages and then started to help others learn how.

It’s probably also worth mentioning here that because Vega and Vega-Lite are not proprietary to Power BI, there’s also many places outside of the Power BI community where you can learn from others. You can refer to Deneb’s support page for details of some of the popular communities where you can network and ask for help.

I’m really stoked that Deneb picked up some really strong supporters at the idea stage and helped convince me that it could potentially work enough to finish-off prototyping it and putting it out there. We also gathered some more enthusiasts during the public preview phase and these folks both helped to promote the possibilities through creating and sharing their own content, and became a great source of feedback for moving Deneb towards GA.

Special thanks here go to Kerry Kolosko, José Escalante and Greg Philps for their early support and great content, and doing the hard yards to help make the languages easier for others to approach in Power BI so early on. Thanks also to Reid Havens and Mike Carlo for inviting me onto their YT channels and presenting to their audiences. Mike has also started and curated a GitHub repository of Deneb templates that others can contribute to, and this continues to grow. We also got some support during the early stages from Power BI user groups, including:

Community: from GA to now

Since going to GA, we’ve seen more people in the community pick-up Deneb and create additional content. There’s been enough traction where I’m now struggling to keep up with what people are doing (which is kind of awesome in and of itself), so here’s some of the ones that stick out:

To everyone (especially those I may have missed), thank-you for sharing your learning and experiences, promoting, and presenting about Deneb and the Vega languages, and for helping others to learn and grow. You’re all truly awesome :)

And, if anyone wants to let me know of anyone who I have missed, or if you want to get their (or your) content added to Deneb’s Resources page, please feel free to let me know. I’d love to host it there if you’d like to share it there.

My experiences of the journey so far (development-wise)

I’ve talked briefly in various places about some of the rationale and technical challenges around Deneb, but I don’t think I’ve spoken cohesively about it and as such I thought I’d dedicate some time to this, as it may be interesting to some.

Rationale

For anyone who has tried to do it, building Power BI visuals using the SDK requires web development knowledge, and while anyone can do this (myself included), it’s a very steep curve if you want to explore a visual idea or have to learn on demand. It’s also a big step taking a visual idea that works well with a static datset and viewport size, and making it much more dynamic, in the Power BI sense of things.

Ever since I found out about the Vega languages, I thought they’d be a great fit for those of us who like to build our visuals in a more declarative manner, and the idea of a runtime environment in Power BI that houses the languages (avoiding the need to build and test them separately) looked like an ideal middle-ground from my perspective. As Vega is quite extensible, there was potential to integrate some of the interactivity features that are expected from Power BI visuals without running into some of the immediate limitations when using R and Python in Power BI.

Reality

Making (what eventually turned into) Deneb turned out to be a lot more complicated than I thought it might be. It’s certainly been a labour of love rather than an entrepreneurial endeavour, and completely researched, developed, and supported during my free time (and for free). As such, there will always be concessions that need to be made to get something out there. If I would have waited until I was happy with it, I’d probably still be writing that first version. And I’m glad I did put it out there, as I’ve already learned so much from those who have been using it.

When I started, I thought that the ideal path would be to leverage work that the existing Vega community had done: Vega Editor is a great tool and it would have been better if I could have just lifted and shifted this code into Power BI. It would have made sense - if possible - to be able to re-use (and ideally contribute to) these tools rather than re-inventing the wheel.

I definitely tried to do this first; however, the realities of making this work within a custom visual became apparent quite quickly. Wanting to ensure that the intended solution could work as a certified visual further compounded this (therefore creating more limitations before I began).

Vega-Editor has pretty much everything I think you need for authoring specifications, particularly as it uses Monaco Editor, which is Microsoft’s editor component behind Visual Studio Code. This has all kinds of neat features - like autocompletion from Vega’s JSON schemas - that are great for productivity.

After a lot of trying to make it work, I deemed it not possible - at least within the constraints of my knowledge at the time - so had to abandon that idea. I’ve tried many times since, including getting help internally from MS on how I could approach it, but it just isn’t possible to make it work easily within a custom visual without having to load all its dependencies from a remote endpoint (and that’s a big no-no for any certified visual).

Because of this, I decided that I would need to focus on creating my own personal interpretation of Vega-Editor, working to these limits.

Retrospective

Looking back at how much I dedicated to researching and developing Deneb over the last 2 years or so, I can safely say it’s been well over 2,000 hours of my personal time to make it happen. My wife has been incredibly understanding and supportive, and I’m extremely grateful to her to have been able to have the opportunity to dedicate to figuring out how the idea could be developed. I mean… I’m still figuring that bit out, and I’m really happy to see that its genuinely helping people do more inside Power BI. I’ve had some great feedback from people all over the world, and I hope that many more discover the Vega languages and can see how they can be applied to Power BI as well.

Deneb has been incredibly useful in my work and for all challenges I’ve had (so far), the Vega languages have been up to the task. I’ve been fortunate to have been able to use Deneb visuals in a number of projects at DiscoverEI, as well as helping other folks (from a professional services perspective) to build their own for a fraction of the cost and effort of starting from scratch.

Through learning and using I have my own list of things I’d like to see, and in addition to this, folks have added their own ideas to Deneb’s GitHub repo. I’m now trying to learn how to wear a product manager’s hat in addition to the development and documentation work; there have been a few big updates since GA, and we’re continuing to move forward.

With each of these new releases, I’ve also been putting a lot of time into refactoring code and endeavouring to find a better editor component that can work in a custom visual, but it just hadn’t been possible to get there yet. I was also finding a lot of the newer features I was looking to do in version 1.5 and beyond were made more difficult by the technical debt that had accrued over Deneb’s (short) lifetime.

I talked a little bit about these challenges in the 1.3 release post, and how I was planning to move forward. This stance hasn’t really changed, but despite saying that 1.3 would be the last significant update, there was enough work to warrant a 1.4 release. I just can’t stop tweaking this thing…

What’s next?

I was hoping to dedicate some space to start talking about where version 2 is currently at, and what it might look like, but this post has run a little long. I promise I will share some details in a follow-up post, but this might be in a couple of weeks, as I’m currently preparing for Difinity, which will keep me busy for the rest of this week and most of next week. Bet you can’t guess what I’m presenting on…

So, for now, I want to say thanks again so, so much to everyone who has found Deneb useful for their needs. I hope that you continue to create great things and will look forward to what’s planned.

As always, thanks for reading. I wish you all the best, and stay safe,

DM-P

comments powered by Disqus