Selecting elements of an array meeting certain criteria help

[frying_pan_cat]

I'm trying to pick out elements of an array for plotting which fulfill a certain condition on another array to plot. I know that probably isn't too clear, so here's an example of how I'm doing it currently

j=1;
for i=1:221184
if(lat(i)>0)
qplot(j)=q(i) ;
psiplot(j)=psi(i) ;
j=j+1 ;
end
end
scatter(psiplot,qplot)

Now I remember reading somewhere that this wasn't the best practice way of doing it but I'm not sure what the best way of doing it is. If anyone can help I'd appreciate it as I'm trying to improve my programming skills and to do things the right way rather just whatever works which is how I currently do it. I'm sorry if this is a stupid/easy question.

 

[Haircut]

Do we have to try and guess what language it is from the syntax?

 

[frying_pan_cat]

Oh, sorry. Thought I'd put that, certainly meant to. It's Matlab

 

[JonJ678]

j=1;
for i=1:221184
if(lat(i)>0)
qplot(j)=q(i) ;
psiplot(j)=psi(i) ;
j=j+1 ;
end
end
scatter(psiplot,qplot)

I think you're looking for:

qplot = q(lat>0)
psiplot = q(lat>0)
scatter(psiplot,qplot) %Scatter gets very slow with large arrays, plot(psiplot,plot,'o') may work out better

That's just playing with syntax though. Matlab is build around arrays of floating point numbers and handles them rather well. Whether that is the right way or not is a bit subjective.

edit: By that I mean using arrays instead of loops is faster and what people who are used to matlab will expect, but that's not necessarily the same thing as "better". Using loops is clearer for people not used to matlab and may handle edge cases more coherently, such as when the arrays have different lengths.

 

[frying_pan_cat]

Thanks, that's the kind of thing I was after. I thought that using loops wasn't the most efficient way but was unsure how else to approach the problem. Thanks for the tip about using plot instead of scatter too.

 

[JonJ678]

Good to hear. Hopefully you're interested in why loops aren't efficient in matlab. The following is an attempt at communicating roughly why this is the case. Please forgive me if it's over simplified.

Matlab is a ridiculously high level language. You type a couple of things in what is essentially English and a load of magic happens under the covers. That's normally great for productivity - I can get an algorithm running sooner in matlab than in any other language I know.

The problem with high level languages is that there's a lot of interpretation to be done before your code gets down to the level where a cpu can do anything with it. That's usually slow. The other extreme is working close to the machine, where the programmer has to deal with loads of fine details (memory allocation for instance). Since what you've written is close to the machine there isn't much interpretation needed before it runs and things can be fast.

That leaves a choice. I could spend an hour writing an algorithm in matlab which will take a day to run, or a day writing it in C which will take an hour to run. That's not great for mathworks as serious numerical work, roughly their target audience, tends to be performance sensitive.

The solution matlab has adopted is to write loads of stuff in the low level languages (mostly C, though I think there's some Fortran in there too) and hide that from the user. A loop is stepping through matlab directly and that's slow. The array notation actually calls some previously compiled C to do the same job and that's fast. This gets called "vectorised code", but it's pretty much just calling a faster language to do the loop.

If you take the loop version and write it in C, you can compile that and call it as a normal matlab function - at which point it'll be about as fast as the array version! So far that's only been necessary once for me - I couldn't vectorise a particular matrix operation within matlab so wrote the C to do it instead.

In case it's not apparent from the above, I quite like matlab. It's a lovely development environment and really beginner friendly.

 

[frying_pan_cat]

That's really interesting, thanks for taking the time to write that out. I think I kind of follow that. Are there any resources you'd recommend for trying to improve my matlab skills for things like this and understand more about how I should be doing things rather than just what happens to work like how I do it at the moment?

 

[D.P.]

A rule of thumb in Matlab, if you have for loop or an if statement then you are quite possibly doing something wrong. Make sure you know your linear algebra. If you use matlab correctly it will be much faster than your hand written c code.

 

[JonJ678]

@D.P. that's a bit heavy handed - if someone can write C or matlab with equal fluency they'll usually get (roughly) comparable performance.

@frying_pan_cat it's nothing to do with matlab, but the pragmatic programmer is brilliant as a summary of generic best practice. If you haven't found it yet, stackoverflow is a useful time sink.

For matlab itself I'm afraid I learned to use it very slowly by trial and error. There's a built in profiler which is useful for working out which parts are running slowly.

The computer science stuff was mostly learned by trying to persuade C to do things though. I'm not clear how one would go about learning to code well using matlab, it hides too much of the nasty stuff from the user.

 

[D.P.]

@D.P. that's a bit heavy handed - if someone can write C or matlab with equal fluency they'll usually get (roughly) comparable performance.

@frying_pan_cat it's nothing to do with matlab, but the pragmatic programmer is brilliant as a summary of generic best practice. If you haven't found it yet, stackoverflow is a useful time sink.

For matlab itself I'm afraid I learned to use it very slowly by trial and error. There's a built in profiler which is useful for working out which parts are running slowly.

The computer science stuff was mostly learned by trying to persuade C to do things though. I'm not clear how one would go about learning to code well using matlab, it hides too much of the nasty stuff from the user.

I don't know anyone who can write linear algebra in C as fast as Matlab's highly optimized fortran/c code which is extremely well optimized and makes full use of hardware specific operators and vector architecture. While writing good matlab code is pretty easy, especially if you come from a math background and you haven't been tainted with c-style programming.

 

[JonJ678]

Matlab is based on the same BLAS libraries anyone writing fortran or c is likely to link to in which case the performance edge is going to be marginal in either direction. Similarly matlab is likely to be built using higher performance compilers than gcc, though I don't know which ones they use.

It all boils down to roughly the same assembler in the end. I suspect fortran -> assembly is still an easier problem for the compiler than matlab -> c -> assembly, but if the matlab engineers are as good as one might hope this may not matter.

"Tainted by C-style programming" is a fascinating turn of phrase. I'll give that some thought, thanks.

 

[D.P.]

I was meaning people rolling their own lin-alg versus using matlab.
If someone does use a BLAS lib then in theory their performance will be close or better IF they can appropriately design and order operations, something Matlab interpreter does very well at.


A lot of people are so used to the c-ctyle syntax that they cannot get a grip of languages like ML/python/F#/Matlab/lisp/prolog that break that.

 

[frying_pan_cat]

A rule of thumb in Matlab, if you have for loop or an if statement then you are quite possibly doing something wrong. Make sure you know your linear algebra. If you use matlab correctly it will be much faster than your hand written c code.

Since making this thread I've been trying to write things more efficiently but I'm slightly unsure how I'd write the following without an if/else, which you suggest is probably the least efficient way to do it

for i=1:timestep+1
    if(x(i)<deltat)
        mheight(i) = 2000*0.5*(1-cos(pi*x(i)/deltat)) ;
    else
        mheight(i) = 2000 ;
    end
end

apologies if it's a really stupid question

 

[D.P.]

Incomplete and non functional (don't have matlab and it's been a long time) but something like this

mheight = ones(time step) * 2000
mheight(x < deltat) = 2000* 0.5*(1-cos(pi * x(x < deltat)/deltat))

At least some thing like that (matlab may precompute the inverse of deltat and multiply the value in the cosine but to be I would do it manually)

You can also do things like

X_= x < deltat;

To get a vector of 0s and 1s for where the condition is true and can incorporate I to the vector equation.
Then you can do element wise operations, e.g y = 2000 .*X_