Feel new job has too much new concepts. (data warehousing)

[413x]

Those who might remember I recently took a new job due to instability in my old situation.
The job is BI and data warehouse (BI being what I know (cubes to dashboard to analysis) and DWH what I don't.

So I took a new job.
In the interview I was very clear that I knew half the job spec. And was totally new to the other half.

They said this is OK. We will train you and you will initially be on the part you know mostly.

I feel now a month in that I'm am much much more on the part I don't know. As well as being in a new company with 1 colleague who is very busy.

I feel I'm being too slow to pick up new stuff. There are a lot of processes which have many steps, many locations and some. Stuff 'you just know'.
My memory isn't great. And my boss saying you just learn this over time is a bit frightening.

The company said they struggled to fill this position and I'm wondering if I've taken a 2 jobs in one position. That my boss can handle but maybe I can't.

I'm mostly concerned with there being just too much knowledge to absorb. Not just in too short a time.. But even once the role is established.
I was wary of this being the case with the DWH side. My boss is being good. But I fear his patience may run out of this just doesn't go in.

I can do the individual elements . (build a Ssis package. Write sql, do the mapping)
But being set on 'here a requirement to bring in new columns.. Go' I just cannot put all the pieces together


Basically.. Have I gotten myself into a position too senior for me.. Despite being clear in interview I do not know DWH stuff.

I bet some of you guys are DWH specialists. And can tell me if I have even a hope of pulling this off?

Also this is quite an expansive (to me) warehouse. Multiple international entities with different erps all merging into one.

 

[Worthy]

So, no, I don't know anything about data warehouse, but what I would say is, you're one month into a new job. You're at the stage where imposter syndrome is common, everything is new, and you're still settling in. I feel like you're being way too hard on yourself. Give it time, you'll be fine!

As a side note - I know you've changed jobs a couple of times during the pandemic. Have these all be WFH? I myself have changed jobs a couple of times too during the pandemic, and because the job changed, but my working environment didn't (same desk at home), I felt the same way, like I should have been settled in and 100% up to speed with everything in a few days. It doesn't work like that, especially in a role like you're currently in.

Stop being so hard on yourself, you'll get there.

 

[413x]

So, no, I don't know anything about data warehouse, but what I would say is, you're one month into a new job. You're at the stage where imposter syndrome is common, everything is new, and you're still settling in. I feel like you're being way too hard on yourself. Give it time, you'll be fine!

As a side note - I know you've changed jobs a couple of times during the pandemic. Have these all be WFH? I myself have changed jobs a couple of times too during the pandemic, and because the job changed, but my working environment didn't (same desk at home), I felt the same way, like I should have been settled in and 100% up to speed with everything in a few days. It doesn't work like that, especially in a role like you're currently in.

Stop being so hard on yourself, you'll get there.

I think if I had a team it would be better. Its just me and my boss. I could lean on multiple people.
Yes I have changed jobs twice! Both for same reasons. Job was changing to something less skilled.

Yeah I feel. I'm being hard on myself. But I feel if I'm too slow as my boss (who is my only support) is so busy it's just going to be a burden to him. Its one of these where I feel I probably need more support than is able to be provided. Or they should have gotten someone with DWH experience and lacking in the BI.

Also. I know technically its not my problem if my boss is too busy. But it also is my problem.

And yes being WFH isn't ideal in this situation

 

[Ayahuasca]

It sounds quite normal in a specialised role to me, particularly in the current climate. I have a friend who has a very specific engineering role and he was headhunted by a small business starting up and he feels the same way atm. The last job he had he'd learned from being an apprentice at 18 and worked there until he was 30 so it was plain sailing. The role on paper usually never plays out how you think it will and some companies will tell you anything in an interview to get you onboard. No interviewer is going to turn around and say "No sorry, we can't train you in those areas but you can still have the job." Training is usually always promised but is rarely given in any meaningful way.

 

[413x]

It sounds quite normal in a specialised role to me, particularly in the current climate. I have a friend who has a very specific engineering role and he was headhunted by a small business starting up and he feels the same way atm. The last job he had he'd learned from being an apprentice at 18 and worked there until he was 30 so it was plain sailing. The role on paper usually never plays out how you think it will and some companies will tell you anything in an interview to get you onboard. No interviewer is going to turn around and say "No sorry, we can't train you in those areas but you can still have the job." Training is usually always promised but is rarely given in any meaningful way.

Yeah I feel like this.
The training can only come from in house resource.
That resource is very stretched.
If I'm taking more time to learn than the company thought its going to cause frustration.

I feel like I was the best of an imperfect bunch.

I'm trying not to just say 'yes I understand' when I don't. Which I obviously have to do. But I do feel this is going to become a problem. My boss is completely overloaded. And I don't think he gets much credit (as is always the case in DWH jobs) for what he does from the senior stake holders




On of the biggest blocks in finding here is.. I can't problem solve my way out.
There are too many unrelated 'just know' steps that if you don't know.. You don't know!

This is probably the first job I feel I can't pick up without basically hand holding

 

[Zefan]

With ref to memory, make sure you're building a second brain. I have learned this the hard way, historically I've been relatively good at keeping track of lots of concurrent tasks/pieces of information, but there comes a point - especially with unfamiliar roles - where this doesn't work anymore. It took me a while to admit to myself that my brain couldn't store all of this stuff itself, but once I did I could then take steps to build systems that could essentially offload things from my brain freeing me up to do focus heavy tasks. You may already be all over this, but the memory part of your post sounded familiar to me.

 

[413x]

With ref to memory, make sure you're building a second brain. I have learned this the hard way, historically I've been relatively good at keeping track of lots of concurrent tasks/pieces of information, but there comes a point - especially with unfamiliar roles - where this doesn't work anymore. It took me a while to admit to myself that my brain couldn't store all of this stuff itself, but once I did I could then take steps to build systems that could essentially offload things from my brain freeing me up to do focus heavy tasks. You may already be all over this, but the memory part of your post sounded familiar to me.

It's my biggest issue. And I feel that's how a lot of DWH people get by. They have a good memory. My boss sure does.

I've never had a role I need a massive memory for. There's always been few processes and lots of problem solving.
This feels like the processes part is much much bigger.

I'm struggling to map these (ie document) in a way I can refer back and actually understand and repeat it.
I'm even trying videoing the calls. But there is still too much missing

 

[booyaka]

Sounds normal - 1 month into a new job/processes/staff etc.

Just do the job to the best of your ability, ask for help when you need it, and just do your thing. You'll be fine

 

[cheesyboy]

I too am about 1 month in to a new role, a rather different one to what I've done before in a company miles bigger than I've previously worked in. The imposter syndrome feeling is real!

Fortunately, it's a project type role (albeit that's something I've no real previous experience with), so I've been able to chop it all into chunks and just try not to think about all the stuff I don't understand yet and will need to think of a solution to later on. Every time I start to think of the whole thing it feels like staring into the abyss!

Just. Keep. Trucking.

 

[413x]

I too am about 1 month in to a new role, a rather different one to what I've done before in a company miles bigger than I've previously worked in. The imposter syndrome feeling is real!

Fortunately, it's a project type role (albeit that's something I've no real previous experience with), so I've been able to chop it all into chunks and just try not to think about all the stuff I don't understand yet and will need to think of a solution to later on. Every time I start to think of the whole thing it feels like staring into the abyss!

Just. Keep. Trucking.

Good luck!
Yeah it's a bit overload at the minute. And I'm doing stuff which could break the entire company!

It's fine before. OK you might break a cube. But that's only a few people impacted.

If I make a mistake on the live warehouse.. Oops.
Obviously theres dev and test. But I'm doing stuff on live.

 

[wesimmo]

If you're concerned, ask your boss for a review of how the first month has gone.

Its easy, as people have said, to go into a new place and feel you should know or have done more, whereas other people may look at you as having done well to pick up what you have in the circumstances and timeframe.

 

[HangTime]

A few points here:

• Agree with the comment about 'imposter syndrome' when changing employer - although I haven't done it many times, I think it normally takes me about six weeks to stop feeling like a charlatan. Half the problem is you don't know what you don't know, or don't know what you need to know until you encounter it.
• You will learn it over time. I moved from BI/Analytics into DWH (although I had some grounding in relational databases, modelling etc) and it's just stuff to pick up. You mention to being an 'expansive warehouse' merging data from different systems, again I had similar when I went from mostly dealing with glorified datamarts over predominantly single data sources to a proper EDW. But that in itself isn't a big deal, and having a relatively robust target schema can help because it gives you some guidance on how to model (if you are doing this yourself, sounds like a small team) based on the existing entities.
• It sounds like your main issue is with design, i.e. you know how to execute different parts of the process but lack the vision of how to stitch it all together. I would suggest finding some worked examples of how previous requirements have been tackled so you can understand a 'default pattern' that you would follow. The good news is if you understand the constituent parts it is then really just a case of breaking down an overall requirement into those parts. You know the processes you need to end up with, so use a worked example to figure out how to get there.
• Maybe ask your boss to undertake peer review at different stages i.e. you take a given requirement and explain to him how you will approach it. Then you create a design and have him give that the once over (maybe combine with the previous step). Then build it and have him review that. It sounds like he's busy so maybe book some placeholder time for this sort of thing.
• Talk to your boss about whether you should have specific focal areas i.e. set yourself targets of understanding X layer of the system by a certain point rather than trying to understand all of it at once. As a manager this might help him because it gives him something tangible to judge your progress against as opposed to just "he's floundering about and doesn't really have a grasp on it".
• Resist the temptation to cherry pick the low-hanging fruit you can deliver easily, as this won't help you in the long run as it will just delay you learning the stuff you will need to know eventually. [Edit: I'm pretty bad at this myself, deferring tasks where I'm not sure how to proceed, and eventually it catches up with you]. The one exception is if you feel it would give a confidence boost and demonstrate to people that you can deliver in more familiar areas.

 

[glenimp617]

My advice as someone who knows a fair bit about this stuff

Good communication, mindset and attitude will see you through. Don't isolate yourself with negative thoughts.

If your manager is seeing a willingess to get stuck in develop, he'll have your back.

Stick with it. It's the people who don't give a crap who get the bullet.

I've just employed two candidates who are in similar positions. I'd rather hire someone who doesn't know it all, who can work great in a team, over a genius who's an ****hole.

 

[tom_e]

It sounds like the bit you're struggling with is the data layout within the companies data streams coming into the DWH?

If that's it and you can do the technical bits then I wouldn't worry, you're only a month in it really is stuff you'll come to just know by exploring and getting stuck in.

 

[413x]

It sounds like the bit you're struggling with is the data layout within the companies data streams coming into the DWH?

If that's it and you can do the technical bits then I wouldn't worry, you're only a month in it really is stuff you'll come to just know by exploring and getting stuck in.

Yes. It's things like 'how do I know this erp system field maps to what DWH field?'
Things like this mainly.
And also interpreting other people's code.

I know understand moving data up through stage, map, warehouse etc.
I get source control and how that works too.
And the process for dev - test - live.
Deploying packages (mostly ssis) to thier respective environments.

But interpreting stakeholder language to DWH and where which erp field goes and most importantly.. Why.. Is hard

 

[413x]

Thanks everyone so far. It is helpful.
Thought be a lot Of people familiar with this stuff.

@HangTime

Boss says he's happy and it will come in time. Luckily he's a nice guy and we get on well.

He's admitted he's behind in documentation etc. Which isn't ideal but it's something we are gonna work on.

It's actually quite logical and neat. It's mainly the erp to DWH I'm struggling with. And people using incorrect or correct (sometimes I don't know) names when referring to things.

I do feel. I'm picking up concepts and the individual elements and it is more 'grand scheme' that's difficult.

I'm definitely not wanting to pick the easy stuff. Ideally. I'll. Work on processes a few times. For the same one to help cement it in. Better to ask for help now than later on harder stuff I feel.

The team is me.. And my boss!

 

[dowie]

I feel I'm being too slow to pick up new stuff. There are a lot of processes which have many steps, many locations and some. Stuff 'you just know'.
My memory isn't great. And my boss saying you just learn this over time is a bit frightening.

The company said they struggled to fill this position and I'm wondering if I've taken a 2 jobs in one position. That my boss can handle but maybe I can't.

I'm mostly concerned with there being just too much knowledge to absorb. Not just in too short a time.. But even once the role is established.
I was wary of this being the case with the DWH side. My boss is being good. But I fear his patience may run out of this just doesn't go in.

Firstly re: the memory thing, get a notepad! Literally, just write everything out - new procedures you've not seen before etc... You're only a month in and they know you didn't meet all the requirements but then again they couldn't find anyone else who did so meh...

Has anyone said anything to you about performance, lots of this might be you worrying too much at this stage, again, you're only a month in.

General tip for these sorts of situations is to be proactive but also not be afraid to ask questions - try to avoid asking repeat questions, if someone has already shown you how to do something basic then write it down in your notebook so you don't need to ask again. Do try to attempt something yourself first (provided there aren't any costs to screwing it up - test, not production etc..) going to someone with "hey, I need to do X, I've tried Y in test I just have a quick query about Z" is much, much better than going to someone with "Hey, I need to do X" having done nothing yourself.

Also, steer your questions towards stuff that is specialist stuff or more specific to the company etc.. basic questions related to some language or software is stuff you perhaps ought to try and solve yourself, or at least partly solve as per previously mentioned.

I don't think it's imposter syndrome as one poster suggested, you know you don't meet all the job spec requirements and they know this too so I wouldn't get overly worried about the fact that you'll need a bit of time to get up to speed on the aspects of the role you didn't have experience of, though it's good to be conscious of it and to be efficient when asking questions or taking up the time of others.

I would put in a bit of effort, after hours or on weekends even, to learn some of the generic skills that might be involved. If your SQL is rusty then use some online resources and sort that out, shouldn't take long ditto to anything else you use - plenty of free or low cost resources, tutorials etc.. out there.

 

[HangTime]

Yes. It's things like 'how do I know this erp system field maps to what DWH field?'

Do you have source-to-target documentation or other lineage docs?
If not you may have no option but to try and understand more about the source system schema. Find out who are the technical experts for it who can help you with this. This is a piece that will definitely take a long time so I wouldn't worry too much about where you are now. The people I had working for me who were strong in this area had worked at the same organisation for many years, I didn't expect novices to know this stuff.
Get yourself some scripts to query the metadata so you can quickly find tables and columns matching search strings.
Also start building your own glossary to track business synonyms and things like that.
Finally there have been times where I've had little option but to take an assumption based approach, conducting data profiling to understand the contents of a given data entity and then inferring where it is likely mapped to / sourced from.

 

[mid_gen]

Tbh, data warehousing isn't rocket science, that part will come to you pretty quickly. Used to be my gig before I got into games.

Ralph Kimball's The Data Warehouse Toolkit cover 95% of the principles required when it comes to dimensional modelling etc.

Domain knowledge is the hardest part to come by. Assuming you're in a permanent role embedded in a company, then you need to get to know the business side, arrange meetings with the people that are consuming the data, understand how the business works.

If you're a consultant, it's a bit more tricky, but you learn to just wing it after a while. I did data warehouse builds for telcos, wholesalers, retailers, banks, all sorts.

 

[OspreyO]

I'm a similar position, more on the DB side though. Except the guy above, never gets time, even after a couple of years. It obvious at this point, he doesn't want to share the load.
Realizing this I make sure to take on other work, that I can do, and keep adding to the skill-set, and have list of high profile projects completed.