[vox] Who thinks Java is cool?

[Norm Matloff]

On Fri, Jun 17, 2011 at 11:14:13PM -0700, Bill Broadley wrote:
> On 06/17/2011 08:41 PM, Norm Matloff wrote:

> > The virtue of the GPU current cards is that you know what you're
> > getting.  People have developed extremely fine-tuned algorithms that
> > depend on knowing exactly what the memory will behave like.  Adding an
> > MMU would destroy a lot of that.  
 
> Hrm, do you think that's really that common?  Sure there are hero codes
> where a shocking amount of work is targeted at a specific CPU
> architecture.  But in general these codes survive for multiple CPU
> generations... especially since writing for a new architecture (like
> GPUs) often means that a new GPU will be out before you are done.  Even
> just today there's quite a variety of nvidia chips shipping, different
> ratios of double DP/SP ratios, different width memory busses, and
> different generations of chips shipping.

It would be interesting to know just what is common.  As far as I know,
there are no surveys.  But certainly the blog posters would indicate
that it is indeed common.  And since it usually doesn't take that much
effort to make reasonably good use of the memory hierarchy and the
interleaving, I would assume most people do so.

> > I suspect the Intel Larrabee will put a major dent in NVIDIA's business.
 
> Heh, I've heard that several times over the years, and heard several
> times that it's back to the drawing boards.  I think Larrabee is dead,
> but certainly I expect something similar from intel eventually.

Well, I don't have an insider's information, but my impression is that
Larrabee is very much alive.  And as you said, Intel will come out with
something to compete with the GPU firms.

Intel will prevail, I believe, for a number of reasons.  I would cite
two in particular (besides its general clout, of course):

1.  The time penalty for communicating between CPU and GPU is a major
obstacle to general performance.

2.  The GPU memory structure is great for you physics/chemistry people,
but not for the biologists, data miners and so on who make up the bulk
of the market (current and potential, as it's still early).  The
physical science market is much too small to drive the technology, as
Cray found out.

> > R has no parallel processing facilities at all.  They're done through
> > add-on packages, the most famous being Rmpi, an R interface to you know
> > what.  My Rdsm package adds a threads capability to R.  But any parallel
> > R is slow on nonembarrassingly parallel apps.
 
> Rmpi sounds promising.

Actually, I've become much less interested in it than before.  It adds
its own finicky nature on top of the finicky nature of MPI. :-)   The
most common questions on the parallel-R online discussion list deal with
Rmpi.  I've personally gone back and forth with Rmpi's author to try to
solve some problems, and some of them remain unsolved.

It's too bad, because the author not only put in interfaces to most MPI
functions, but also included some clever extensions.  But I've basically
given up.  I've taken Rmpi out of my book, other than a mention that it
exists and is versatile.

For most users, another parallel R package, snow, is much better than
Rmpi.  It basically does a scatter/gather operation, and in a very
simple, convenient manner.  It's almost TOO easy.  Rmpi adds value only
in applications in which lots of nodes need to interact directly with
each other rather than with a manager node, which is not common in the R
world.

There are other message-passing parallel R packages too, such as foreach
and multicore (not what it sounds like).  But they are much more complex
than snow, without much benefit in my view.  (I say that even though I'm
close to the company that developed foreach and made it open source, a
very good firm.)

But of course, my own bias is shared memory, for clarity's sake, so I
like my own package, Rdsm. :-)

> > My next question is whether the goroutines give you true parallelism.
> > Or can only one run at a time, which the term "coroutine" would seem to
> > imply?
 
> Yes.  On my embarrassingly parallel mandelbrot set routing I get
> basically perfect scaling.  Not sure exactly why they are goroutines,

I guess they couldn't resist the pun.

Norm