Programmers?

[dangerstat]

That's what i ment by cores, as in cores in a cluster. If you can spread across core you can spread across clusters, it just the matter of getting the state over to all the boxes, which many programs will do for you.

It's just getting it parallel in the first place, which is hard.

The two are different though, things that can work well over clusters do not necessarily work well over cores. It really depends on where your hitting your hardware limits. Typically with a cluster environment you'll hit network bandwidth issues where you simply can't transfer data between the nodes as quickly as you'd like, in a one machine multiple cores setup that's not really an issue.

But there are implicit problems with map reduce in practice. It'll fall over where in the map stage if the records are highly asymmetric , or equally if the files sizes themselves differ massively. But the ultimate point is that IMHO it's inefficient for most applications

 

[UK|AL]

The two are different though, things that can work well over clusters do not necessarily work well over cores. It really depends on where your hitting your hardware limits. Typically with a cluster environment you'll hit network bandwidth issues where you simply can't transfer data between the nodes as quickly as you'd like, in a one machine multiple cores setup that's not really an issue.

But there are implicit problems with map reduce in practice. It'll fall over where in the map stage if the records are highly asymmetric , or equally if the files sizes themselves differ massively. But the ultimate point is that IMHO it's inefficient for most applications

You hit bandwidth issues in any cluster situation though, but you don't hit that often, or you use a high bandwidth cluster solution which there are many.

Are you talking about the amount of records being aysmmetric and splitting up the work might be a little one sidded? i'm pretty sure no cares if a few machine runs a little longer than others. You will have to be a little more explict on what problem is for me to understand what your getting at.

Scaling with map reduce has the capbillity(In theory) to be linear, as in you add 1 machine it doubles, then add 2 machines it doubles etc. It's pretty efficient in horizontal scaling which is what really matters in parrellel processing.

 

[Range]

Java/C++!

 

[mattgaunt]

I could be completely wrong but isnt the map reduce the algorithm behind Hadoop? or what ever its called, the open source algorithm yahoo is using now?

If it is I had a talk from a guy from HP who explained it to me and basically was saying how its great for when machines in the cluster are down as it doesnt prevent any of the other queries getting delayed.

I dont really see a much more efficient way of doing this?

As for the C and Haskell debate, I do find with haskell when it works, it works, C it doesn't but I would choose C over haskell any day purely because of the higher level of C. But it would be interesting if a game was ever made in haskell.

 

[nicl]

Thought I would break up the flow a bit! I am learning PHP. That's it.

So take your clusters and high brow discussions because I don't understand any of it. I'm learning the basics. Hooray!

But having said that, clearly [insert random programming language] is better because it handles jizzrythmic algorithms and clusters etc. in the most esoteric manner.

*edit* today I learnt about nested loops!

--

<?php
echo "<table style=\border: 1px solid black;\"> \n";
for ($y=1; $y<=12; $y++) {
echo "<tr> \n"; // tr means table row - so for each iteration a new table row is produced and populated
for ($x=1; $x<=12; $x++) {
echo "<td style=\"border: 1px solid #000; width: 25px; padding: 4px; text-align: center;\">";
echo ($x * $y);
echo "</td> \n";
}
echo "</td> \n";
}
echo "</table>";
?>

--

 

[dangerstat]

Thought I would brake up the flow a bit! I am learning PHP. That's it.

So take your clusters and high brow discussions because I don't understand any of it. I'm learning the basics. Hooray!

But having said that, clearly [insert random programming language] is better because it handles jizzrythmic algorithms and clusters etc. in the most esoteric manner.

*edit* today I learnt about nested loops!

--

<?php
echo "<table style=\border: 1px solid black;\"> \n";
for ($y=1; $y<=12; $y++) {
echo "<tr> \n"; // tr means table row - so for each iteration a new table row is produced and populated
for ($x=1; $x<=12; $x++) {
echo "<td style=\"border: 1px solid #000; width: 25px; padding: 4px; text-align: center;\">";
echo ($x * $y);
echo "</td> \n";
}
echo "</td> \n";
}
echo "</table>";
?>

--

What do you want? A cookie?

 

[~J~]

So take your clusters and high brow discussions because I don't understand any of it. I'm learning the basics. Hooray!

Psssst, most people here are in the same boat as you, they just don't have the balls to admit to it

 

[dave-lew99]

What do you want? A cookie?

Hey! They've all got to start somewhere! Give him a break.

 

[nicl]

What do you want? A cookie?

I think you've misunderstood the attempted humour in my post.

Now that you've forced me to get serious: my point was to suggest (in a light hearted way) that people are getting a bit over the top in their 'x is better than y' discussion, not to suggest I am really proud of the fact that I am unable to program even the basics.

But yes, to answer your question, a cookie is always tasty.

 

[Cuchulain]

I do PHP & Javascript, nothing else though I'm going to have a bash with Ruby and Python very soon.

HTML and CSS are not programming languages.

I used to be a COBOL/JCL (though JCL is only for scripting mainly) programmer, on an ICL ME29 18 years ago!

 

[mattgaunt]

Even the hardcore programmers like cookies

 

[~J~]

I used to be a COBOL/JCL (though JCL is only for scripting mainly) programmer, on an ICL ME29 18 years ago!

Jeez that's a blast from the past, was a Quattro user myself (model 39), sure don't miss COBOL.

 

[dangerstat]

I think you've misunderstood the attempted humour in my post.

Now that you've forced me to get serious: my point was to suggest (in a light hearted way) that people are getting a bit over the top in their 'x is better than y' discussion, not to suggest I am really proud of the fact that I am unable to program even the basics.

But yes, to answer your question, a cookie is always tasty.

I was only messing

How are you finding PhP? Not a fan here. It's so similar to Perl that it might well just keep Perls syntax

E.g.
PhP
<code>
foreach ($bits as $bit){
print $bit;
}
</code>

vs.
Perl
<code>
foreach $bit (@bits){
print $bit;
}
</code>

Argh! drives me insane!!

 

[nicl]

I was only messing

How are you finding PhP? Not a fan here. It's so similar to Perl that it might well just keep Perls syntax

E.g.
PhP
<code>
foreach ($bits as $bit){
print $bit;
}
</code>

vs.
Perl
<code>
foreach $bit (@bits){
print $bit;
}
</code>

Argh! drives me insane!!

Hey, I am finding PHP ok, but literally just started recently and have not done too much yet. Also, this is the first language I have tried to learn so I don't really have much to compare it with. To be honest I'm only learning it so that I can effectively use CMS's like drupal better.

If anyone is interested I'm using the book 'Sams Teach Yourself PHP, MySQL, and Apache. So far it seems a good intro into all three.