Just One Data Point, But…

Sorry for my lack of posting for a while. I have a ton of material to discuss, but just don’t have the time. But I couldn’t resist posting this one. Bear with me, as it takes some setting up.

Some of you know, or know of, Mark Regets, retired from the National Science Foundation. In his heyday, he was NSF’s biggest booster of the H-1B program (a high bar!). Well, both Mark and I recently got into using Twitter. I’ve had an account for years (@matloff), but didn’t really use it until an unrelated issue came up.

A week or so ago, journalist Noah Smith (@Noahpinion) and I were discussing PhD production in the US. Though Noah has been a big promoter of H-1B in his Bloomberg View column, the conversation wasn’t really about H-1B or foreign students. But I did state that we are overproducing PhDs, and in jest I wrote “Look at all those PhDs working as barristas!”

Well, Mark saw that, and said this a myth, the PhDs are mostly in decent jobs, etc. I replied that it was just a joke, but seriously many PhDs are UNDER-employed.

I am quite active in the R programming language, which is widely used in the data science field. The annual worldwide R conference, useR!, is currently in progress in Brisbane, Australia. One of the attendees tweeted, I believe seriously, the following:

BARISTA: What are you doing today?

ME: I’m at an conference .

B: Oh I love R, I just moved from SPSS and Stata!

(SPSS and Stata are two commercial products that are fast losing ground to the open source, better quality R.)



22 thoughts on “Just One Data Point, But…

  1. My son and his partner both have PhD’s in neurophysiology. My son is underemployed and his partner recently left science after a prolonged post-doc. Both are trying to remember why they decided to get PhDs.

    I participate in workforce groups where employers claim they can’t find workers. Inevitably, for each such employer, one parent or another in the meeting says their child has a tech degree and is living in their basement, while trying to find work.

    Liked by 1 person

    • Interesting about those workforce groups. Our Bay Area R Users Group, has monthly meetings with speakers etc. We open each meeting by asking who in the audience is looking to hire, and who in the audience is looking for work. To my knowledge, this has never resulted in a hire, or even an after-meeting chat between an employer and a job seeker. No Purple Squirrels attend our meetings, I guess. 😦

      Subsequent to my making this blog posting, there was another tweet from a useR! attendee, saying that on his way back after the conference, he ran into a train ticket seller who turned out to be well versed in R.

      Liked by 1 person

  2. OK then, let me try going almost completely off-topic then – hey I’m driving around a new Honda that has the MobilEye ADAS package of driver-assist features, only a baby brother compared to what’s on a Tesla, but even so, all of these things make various claims to be using AI and ML techniques.

    Norm, do you have anything you’d like to post about the new automotive systems? However tenuously related to your usual topics?


    • I don’t like that new-fangled stuff. Seriously. I don’t even like ABS. And don’t get me started on indirect “emergency” brakes.

      When the “autonomous vehicles” first came out, I was talking to someone at Google who was raving about them. I said, “Wait until someone gets killed,” which sadly has occurred.

      Liked by 1 person

      • Norm, I think I told you and others that I deeply worry about software correctness especially with dynamically type checked language. I strongly am an advocate of statically and strongly type checked languages like http://www.haskell.org. I think R is in the category….where functtionald and imperative parts of a program are strongly segregated. I digress….

        Autonomous and dynamically checked software really scares me. I have known too many “macho” out-of-control software jockies….overconfident. They need a very tough statically type checker …

        How about autonomous battle robots …does the film T2 ring a bell.

        Sorry ….can’t find my glasses.

        Kind regards,



      • I’m undecided (but what on Earth could you have against ABS brakes?!?) on the actual stuff, I’m enthusiastic about the project, I’m bemused by what seems the extremely quick progress over the past few years, I’m hopeful they do get to full autonomy by the time I reach age 72 and that I can afford it – or that autonomous taxis are then so common that I don’t even need to own a car anymore.

        I think LA urban traffic will be immensely helped by advanced systems, I could go into detail. I’m just asking about the current systems, their design and architecture and the mathematics behind them. A lot of people seem very happy with this MobilEye system that’s in about ten or twenty different car lines now, but when I drive it sure makes a lot of errors.

        You should give it a try, Norm, you can have fun reverse-engineering it as you drive, to try to anticipate what it will do next, and why it just did whatever it just did!


        • I actually have driven by Google autonomous vehicles. I live in the Bay Area, after all.

          I’m skeptical about these things because I know how software is designed.


          • Sure, but in principle the questions of cognition and AI don’t depend on craft issues. The question of how to best pursue the craft are related, of course. I think there’s a lot of room for improvement in both and nothing prevents much better work being done.


  3. In the last 30 years, the use of H-1Bs, OPT, F-1, J-1, and TN-1 has resulted in a vast inflation of the science workforce. The biological sciences are HUGELY overstaffed. In 1980, a post-doc was advised for biological sciences. I had never heard of post-docs for statisticians, but was offered one. Today, you must have 1 post-doc, sometimes 2. Many people end up in post-doc careers as essentially over-qualified lab techs never able to get to the tenure line position.

    We need to get rid of the OPT entirely. It has gone from 40K/year to 300K/year. There were 300,000 OPT visas. That is the full graduate classes of Illinois, Michigan, Wisconsin, Indiana, Tennessee, Missouri, and maybe 15 more schools. These OPT workers take the CRUCIAL first job from American kids. AND THEY GET A TAX BREAK FROM THE FEDERAL GOVERNMENT!!!!!!! It’s cheaper to hire an OPT than a US citizen.

    Many of these are Americans who cannot find jobs. I will not hire any foreign worker on a visa.

    Liked by 1 person

    • If there were no foreign students, the number of PhDs would increase. Whether the resulting number would be more than what is suitable is not clear.


      • I think that the problem of PhD overproduction is different from H1-B/OPT problems.

        After all, PhDs were being overproduced as far back as the late 1970s.
        (That is a major reason why I did not get a PhD, despite Yale offering me a slot.)

        At the time, people spoke about “Academic Birth Control”. πŸ˜‰

        The central problem with PhDs is that many professors think that they should
        have a bunch of grad students, which tends to cause the population of PhDs to
        increase in an exponential fashion. This is a common situation in ecology.

        Maybe what we need is more predators? 😐


      • Norm, have you ever heard of any college or university warning the parents of citizen students not to go into STEM careers? That’s what I’m expecting.


      • I think you misunderstood. I meant “If no foreign students graduated from US universities with PhDs, how much better would the underemployment situation be for Americans who graduate with PhDs? Could you put numbers to it for EECS and statistics?”

        In my field, I feel that it would eliminate the need for post-doc positions for Americans looking to eventually land faculty positions, and also ~ double their average PhD stipends, as about 1/3rd to 2/3rd of PhD students are foreign, depending on the university.


        • The post doc situation is shameful. The whole idea of post doc positions was to tide some new PhDs over in a slow job market. Now it has become nearly mandatory, due to the oversupply. So, yes, ending the F-1, J-1 and H-1B programs would help a lot.


  4. I would like to inform readers that R is available in SageMath. SageMath is fun and easy to use. SageMath is a free open-source mathematics software system. In the Linux version, just type “notebook()” at the command line prompt to get to the GUI.

    Perhaps the niche software market gives universities a sweet-heart deal hoping that students will learn and then use their software in the real world.

    I would also like to inform readers of viXra.org (that’s arXiv spelled backwards). There is a statistics section. There is a lot of debate about academic papers being published in pricey journals instead of being freely available to the public, especially for government backed research.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s