PowerPivot database woes

Soldato
Joined
2 Aug 2004
Posts
8,041
Location
Buckinghamshire
I've never been great with database management, but I'm trying to pull together a refreshable standardised report where the end user can select a time frame to look at.

I'm bringing data into PowerPivot by:
  • Exporting data from a server (via Microstrategy in Excel)
  • Manipulating the data via VBA then saving to CSV
  • Import the data via a PowerQuery into PowerPivot (I transform data so I can split one column such as "Value Sales LY" into two "Value Sales" & "LY" - so I can use these as slicers)
The data basically has product sales by month, with each month represented by the first day of the month (eg 01/01/2019 for Jan 2019). There's a column for facts (Unit, volume and value) and another column for if the fact is for this year (TY) or last year.

What I want to be able to do next is have a column that includes (so I can have a slicer that changes the data based on looking at latest full year/MAT & latest QTR):
  • Latest MAT (the latest 12 months, working back from the latest month in the data)
  • Latest QTR (the latest 3 months)

I've tried creating a new table with two columns
  • One column with all the months
  • The second with a descriptor (i.e. 'Latest MAT' against all the months)
However I ran into a many-to-many relationship error as this obviously has duplication with months, e.g. Dec 2020 is part of Latest MAT and Latest QTR


I then tried:
  • Creating a 'selector table' which lists Latest MAT, Latest QTR and Latest Month
  • Then three tables such as Latest MAT with all the months against it, Latest QTR with the last 3 months against it
  • Then created a relationship between the selector and the three tables
This then falls down when I try and link the three tables back to the main data table but get this error:



Database tables here:


Any support much appreciated.
 
The "TIME" column in the CSV contains the date, the data is monthly and each month is represented by the 1st of each month. I.e. 01/01/2020, 01/02/2020.

I could just add in a 'TIME' slicer, but was trying to make this document as simple and unbreakable as possible to stop people getting the wrong insights and date out of it.

Tried Googling this all night but keep hitting brick walls. A lot of tutorials are using PowerBI and/or it goes over my head. Even tried the column add function using the calendar function, but can't even get that to work.
 
Really appreciate the support guys and taking time to reply.

I've addressed your points below.

Maybe I don't understand what you are trying to do. I don't know why you need extra tables here. You only need one. The CSV.
I would just create two columns in the CSV one MAT and the other QTR. Populated as appropriate.
Then create a power pivot against this now "flat" data.
MAT and QTR would be "different" slicers. One would have MAT/NULL the other QTR/NULL, the data will overlap. Since it exists in both

Personally
I'd prefer there was slicer for year then month. I often actually create a column with Month (February etc) and another Year (2021) and have slicers on them.
Let them select the dates themselves.

Because MAT or QTR will change every month, are they really needed?

But if I create columns then I can't use a slicer as slicers show choices within one column/field.

Whilst I see what you're saying about two slicers, a slicer for each month would be cumbersome because you will need to select multiple items in two slicers. For example, the MAT based on January 2021 would require selecting 2020 & 2021, with month containing a multiple selection of Feb through to January.

All I can say is trust me when I know the limitations of my colleague's ability to translate MAT into selecting the right slicers.

This query is right up my street, I'm a former MicroStrategy developer/architect, PowerBI/Qlik/Salesforce/Tableu third line support and an ETL programmer.

Power BI used to not detect dates and hierarchy them into year, month quarter, date etc.
Back in the day most people would keep a good advanced query to generate a big comprehensive date table, then relationship the source into it and use the date table for all time slicers.
Excel and Power BI are pretty much the same in the query editor and relationships so the approach works in excel too.

Grab an old date table script from the net, one where you define an earliest date and final and it populates all possible dates.
This date list would also have columns to define the date in all possibilities like month name, month no, day name, day no, year, year-month, quarter.
Etc. Relate your source table into this and then use the date table as a slicer.

Edit:
This page looks decent
https://blog.enterprisedna.co/how-to-create-a-detailed-date-table-in-power-bi-fast/

I'm unsure of your MAT and QTR requirements and what this means but if you can work them into a date table column using a case/conditional statement then jobs good.
If this is some kind of filtered start-end accumulative total count then it's complex but could involve something like a dax column insert with some nice formula that will take ages to work out.

You're reinventing the wheel a bit though. You're using a federated enterprise grade reporting platform (MicroStrategy) that has data control and top tier ETL capabilities then exporting it to excel to use as an interactive report.
Excel is good for small report data sets when you need to power query for some simple ETL without comitting to a BI platform but you should use the excel data in a BI platform for user interactivity.
This will give nice slicers, aesthetics and lock them out from breaking things.


Second edit.
Read the question again.



You're using visual basic in excel to transform the data?
I once did that and I'd recommend against it, when you get promoted you can be hounded a bit to fix anything overly proprietary.
Power query is all standardised functions whereas VBA can be proper coding.

MAT
If the 12 months is rolling (ie 12 months from current month) then get the current date and evaluate each row's date and apply values in a new conditonal column.

QTR
Same type of thing, define the months in Quarter if different to standard financial definitions then create a column.

There's many ways to skin a cat, do this in vba or transform it in the query editor (DateTime.LocalNow)

You could also do without a date table as someone mentioned above and do this all in one flat de-normalised table. I'd still recommend a date table though as its a good practice to denormalise things that are absolute like dates and may apply if you ever need to integrate a second data export for a different thing in a dashboard.

If you're looking for a neat visually appealing and interactive plaything for showing off data then try Power BI. It's essentially excel mixed with powerpoint and you can export a multi dashboard page PDF with mixed reports and narrative elements. Use power query to do the data transformations, even though Power BI can run everything verbatim from the power query it's good to keep this separate otherwise you need to change multiple reports every time you modify the data structure.

Just to address some of your points:
PowerBI - Unfortunately I don't think we use PowerBI - it's not on my machine and not on our Software Centre. We use Office 365 on semi-annual Enterprise and the icons under 'this product contains' show the typical Office applications such as Word, Excel, PowerPoint, Outlook, OneNote, Publisher, Access and Skype for Business.

VBA - I know some of what I'm doing in VBA can be achieved during the PowerQuery or PowerPivot, but I'm not sure if all of it can. One of the things I'm doing in VBA is creating a 'future' sales data set, i.e. taking the last 12 months data, duplicating it but adding a year onto the date and changing the "Value Sales" etc to "Value Sales YA". This can probably done with PowerQuery but I'm a basic user and can achieve this in VBA which I'm far more comfortable with.

Microstrategy - Unfortunately the data service where the data is pulled from isn't in house and I can't see any options to run advanced reporting. I can see two reporting options, adhoc and trended reports. Adhoc is hard set to only showing the absolute latest data by MAT, QTR & Latest Period. Trended splits it out by month.

You're probably wondering, well if you can get the MAT & QTR reports why not just use those? Well I could, but that would require importing two reports - it's a potential option. The reason I have monthly data is the report serves two functions (i.e. I see it as efficient to have one data source that just needs rolling up)
1) Monthly pivot chart where users can select via slicers to see performance YoY, either looking at total sales or at certain segments and looking at different facts (Packs, volume and value). This is already up and running and working very well

2) Is a roll up of market shares for the total market and retailer level. We only ever look at this MAT and Latest QTR and would like to look at our share at different levels. I.e. our share of volume, value or TDPs on either the MAT or 12W
 
I have potentially found a solution but haven't had time this weekend to put it into practice. Not entirely 100% on the understanding of the solution, it was posted online somewhere as pseudo code but I think I roughly understand what it's trying to do.
 
Well bad news is the solution I found doesn't work:
https://www.mrexcel.com/board/threads/create-period-slicer-in-powerpivot.936104/

I get the principle (I think), but their post is a little confusing as they run a min function on the timeframe ID but that measure (SelectedOption) isn't used in their solution. Plus using a slicer to control that unconnected table doesn't change the SelectedOption measure, it always equates to 1 if you have 1 to 3 as your timeframe ID (1 for MAT, 2 for QTR and 3 for Latest Month).

On the plus side I've built some incredible tools with PowerPivot this last week so I've learned a lot already. Might just admit defeat on this one and use a time line filter.
 
Sad to hear that mate. I kind of knew your likely route to salvation was going to stick to a data then slice that:

See my last post, I manged to find a resource online in the end which worked. :)

It's basically taking a disconnect table, returning an ID using a custom measure with a min function on an associated ID number to my timeframe description such as MAT, QTR and Latest Month.

Then a switch statement that changes a custom measure to MAT values if the ID number = 1 etc.
 
Back
Top Bottom