Best way to display large amounts of data

[balanceballs]

I have a database table with about 20 columns and need 3 of them to be editable. I've stuck it all in a Gridview (.NET) and it kind of works but is a bit of a chore both to use and in terms of maintenance. The sheer amount of code I've had to write just to get the thing looking ok & to be able to manipulate the data is staggering. I still haven't implemented anything to filter the data.

Is there a framework I can plug into a .net page to handle data for me? Surely there's little point re-inventing the wheel? Gridview seems quite limited.

Apparently the users will require visibility of ALL columns for some reason. What's a good way to display this so you're not constantly horizontally scrolling?

Thanks

 

[balanceballs]

I have approx 80k rows so far but this will increase over time.

It's a .net webpage yeah. Is there a way to show some data and have a + or an arrow to show further info just below?

 

[balanceballs]

Check out DataTables. I'd avoid all of the extended .Net web controls as they tend to be complete crap. All you need to provide to the above is an endpoint which returns paged and/or filtered JSON. Users can then sort, show/hide and rearrange columns as they require, eg. https://datatables.net/examples/api/show_hide.html

This is actually what I'm using at the moment. Unfortunately it doesn't play too nicely with gridview. I'm not sure how to get my data in JSON format. I had a quick look and it seems overly complicated. I haven't got a lot of time to get this done which is why I was looking for something open source that would kind of do it for me. I've been unable to find anything yet that meets the criteria.

I last built something with ASP.net a few years ago because I've switched to PHP. However, that said, I'm sure that the latest version of ASP.net will have lots of controls that can be used to display data nicely, with paging (which will be necessary for 80k rows!!!)

Is it that you are relatively new to ASP.net and don't know what's available, or is it that you've looked hard and don't like what you've found?

As well as all the standard controls offered by ASP.net, there are also a number of specialist .net companies that offer data controls, Telerik.com springs to mind.

Rgds

I've implemented paging and sorting so far, as they're built into the gridview and seem to work reasonably well. Currently trying to get some sort of searching or filtering to work, then I'm done, hopefully.

I'm quite new to .NET yeah and used to be a php developer many years ago but have long since forgotten just about everything. I'm not necessarily tied to .NET but it suits the environment I'm working in (as in - the dev environment is already configured). I'm in quite a junior position but have a considerable amount of responsibility. I'm also limited by time.

The problem I'm finding with even trying to create a simple application is that there's so much to learn. It's not just C#, it's HTML, CSS, Javascript (and all the libraries/variants), SQL and the whole .NET framework - all this entity, linq, ADO stuff is just confusing me. Then there's MVC, web forms, etc to further complicate it. The code can be so bloated as well.

From my research on this so far I've found perhaps 10 different ways of doing things, each of which has its limitations and each has stumbling blocks where I spend hours trying to figure out why I'm encountering a problem.

I've seen quite a few paid for options like Telerik, devexpress, etc. Even looking at the usage examples for these though, it's not entirely easy! Plus I won't get approval to purchase anything at the moment.

What I'd really like, to save my sanity and allow me to get on with some other stuff, is something where I just say - the data is in this table, make these columns searchable, these ones sortable, then it just goes and does it. I'm really surprised it doesn't appear to exist!?

 

[balanceballs]

I'd stay away from web forms if you can help it. ASP mvc is great, web forms not so much, although a lot of stuff was done originally in web forms as it's quite RAD and blackboxing, so probably hard to do so if you have existing projects.

I wouldn't worry about Sql and ado.net for now, entity will do what you need. Baby steps.

I've tried MVC before and got completely lost. I'll have to give it another go but it's just finding time. It's ok running through tutorials and stuff but then trying to apply it to something in the real world, I get stuck and spend my day googling how to fix something minor :/

 

[balanceballs]

I understand exactly the kind of situation you are in re the difficulties of working with ASP.net, and that's a big reason why I switched my focus to PHP, because I found examples and help a lot easier to come by. With ASP.net I found it hard to find easy to follow examples, and if I used an "off the shelf solution", I would often find limitations that I couldn't overcome.

If you have been set to do something in an unrealistic timeframe, it is best to explain the tech learning curve you are facing as best you can and say you need a bit longer.

That said, as fair as your criteria is concerned, whilst its years since I did it, I have done something similar to this. I used a DataGrid control and the Paging control offered by .net. You may have to resort to using a "plainer" control which you then apply custom coding to if that makes sense. At the end of the day, I always found finding a similar example the most helpful, some of these sites might help you find something. Failing that, maybe reach out to one of the ASP.net developers pros and see if they will help you.

Yep, it's a nightmare, however today I knuckled down and got it done! It took a while and I got stuck a lot on some bits that probably should have been easier, but my page works properly. I can search multiple columns and have achieved what was asked of me.

I did explain to my boss that I was finding it difficult - he understands and supports me but also has to turn out the work. We're a very small team (4 IT staff supporting about 200 users) and I'm technically the sole developer at the moment. I was promoted from the helpdesk because I showed some talent in this area so my boss is well aware of the limitations. I think because I've managed to muddle through things and learned some of our more bespoke stuff inside out, perhaps it comes across that I'm more knowledgeable than I am.

I should add, if you use Discord, there is a C# server with lots of helpful people around who answer any question quickly although I am going to take a guess you can't use Discord browser client at work? Someone might ask questions if they saw you on it haha?

I can use whatever I like at work to be honest, so I'll have a look at Discord. It looks a bit like Slack, which I've used in the past.

Web forms is just horrible.

MVC is a dream compared to it.



A webapi controller will write JSON out with little effort, but that's another technology for you to look at.



As someone mentioned above... do it in baby steps. Have a look at some of the tutorials on the http://www.asp.net/mvc site. I think it's the MVC / Entity Framework one that I found especially helpful when learning MVC.

My suggestion would be to use MVC get the data you want out and into a basic HTML view, then add the CSS to dress it up and make it look presentable.

MVC appears to be awesome from what I've seen. However, when you only just understand OOP, trying to make sense of the whole model, view, controller and apply it to anything outside of a tutorial is very difficult.

The tutorials on asp.net are ok but as I mentioned, it's sometimes difficult to relate them to real world applications / data. I did the full 'Movie' website tutorial and am still none the wiser on some aspects.

What I did in the end was to expand on my gridview (because for the most part, it was working), using things like TemplateFields, ControlParameters and the Datatables thing mentioned earlier. Only took me all day and 200+ lines of code, but I'm there!

 

[balanceballs]

Sounds to me with that much data to view and edit you probably need a client side app. Wpf or even win forms. It would be a doddle managing that much data.

Then again also sounds to me that your requirements are mixed. In all honesty if somebody needs to see that many columns then dump the data out into a spreadsheet, capturing information then could be done in a simple Web app.

It's not that they need to see it all at once, they just eventually need to go through and edit 3 columns in most of the rows. It's for an archiving process.

I've stuck a massive search box at the top which searches some of the columns for them, then shows a max of 15 records. This should reduce the hit on our database by doing the filtering at the SQL end and it seems to work pretty well in testing so far!