The Lounge - CodeProject

Depends on the language.

CQ de W5ALT

Walt Fair, Jr., P. E.
Comport Computing
Specializing in Technical Engineering Software

Hi,

Just wondering, does anyone know which are the most used tools for statistical analysis?

I know about R and Python(with pandas), but there has got to be more than that... I would imagine that oracle or microsoft would have something, but so far I haven't found anything.

To those who work with such tools, what do you use at your company?

Regards

There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell

R is more of a programming language aimed at statistics.

I believe the two main competitors are SPSS[^] and SAS[^]

But if you google for either of them you'll get more ads/links to other programs.

Wrong is evil and must be defeated. - Jeff Ello[^]

Hi,

Thanks for the tip I'm looking at the sites of both SPSS and SAS and this is the kind of thing I was thinking.

I came across this area not long ago and found it to be very interesting.

Would R be the main one to learn? Have you got personal experience with it?

Regards

There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell

SAS and SPSS are tools, R is a programming language (of sorts), so it's mostly up to what you want to achieve.
Keep in mind that R isn't intended to be anything but a specialized language that never will be mainstream.

Also note that I have no personal experience with R, I've only read up on it a bit to see if we have any use for it.

Wrong is evil and must be defeated. - Jeff Ello[^]

I worked extensively in SAS... it was dreadful. If you care to know why, I can elaborate. I won't go back to it.

We're having a couple of people here that uses SPSS. They're not happy about it. It suffers badly from featuritis while lacking usability and stability.
So yes, I'm interested in hearing your opinion on SAS.

Wrong is evil and must be defeated. - Jeff Ello[^]

I used it about 3 years ago... I worked with it for a about five years up to that point (versions 8 through 9.1).

1. Licensing was always an issue. They were stuck on some old mainframe idea of charging per processing core. If you tried to run it on a modern Linux or Windows box it became unaffordable without a long negotiations fight.

2. It requires a mind shift to realign with their programming practices. All the SQL you might know and love is backwards in the world of SAS, which processes everything more like a cursor. So while that is not necessarily bad, it is uncomfortable until you understand it.

3. The interpreted sas language, which was powerful and useful for its original design (creating massive reports) was terrible if you had a real-time transactional piece inputting or updating to the database. Everything runs best in batch. You will require a 2nd system for transactional input, and require a daily synchronization process at night.

4. The interpreted sas language itself was inconsistently implemented. The syntax around keywords and operators, even for simple things, could be different from one feature to the next, so you were always having to look up documentation (even after 5 years).

5. The compiled SCL language (which I think they were dropping support) had compiler issues. Even adding or removing white space could cause random errors (basically the grammar had bugs and was unreliable). You could spend days tracking down compiler problems in and around SCL.

6. Things that should be easy in other languages are difficult or expensive (additional feature, new license) in sas.

7. Documentation was regularly missing, and unlike more popular languages, I couldn't find what I needed on the web. Maybe that has changed with some of the newer social Q/A sites.

I have a lot more opinions, but those might be related to where I worked, and not the product itself.

Edit: I had a very bad job at the time (on call every other week, multiple calls per night, etc, etc). A bank called and offered a job where I would convert SAS into .NET... I refused because I was not taking another job dealing with SAS Dead | X|

It was really that bad.

Have some sympathy and an upvote for the info.
We were quite recently recommended by a professor in statistics to "upgrade" to SAS, I'll think we'll pass.

Wrong is evil and must be defeated. - Jeff Ello[^]

I was also going to refer you to SPSS - I know multiple universities in Sweden at least have site licenses and use it for statistical analyses in their research. Very pricey for an individual though.

When looking for a replacement for the Mrs (for the software she wants, not her!), I also came across PSPP, an open source replacement for SPSS.

Iain.

I am one of "those foreigners coming over here and stealing our jobs". Yay me!

Oh, thanks for that tip! Thumbs Up | :thumbsup:

Wrong is evil and must be defeated. - Jeff Ello[^]

Iain Clarke, Warrior Programmer wrote:
PSPP, an open source replacement for SPSS.

Is the P<->S change deliberate ?

~RaGE();

I think words like 'destiny' are a way of trying to find order where none exists. - Christian Graus
Entropy isn't what it used to.

Rage wrote:
Is the P<->S change deliberate ?

I'm sure it's a complete coincidence. Nothing to see here, move along...

Iain.

I am one of "those foreigners coming over here and stealing our jobs". Yay me!

Open source.... Probably one of the best ways to learn about something!!

Thanks

Any recommendations on what else to learn to better work with statistical analysis?

There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell

I told an analyst at our company about PSPP, and her comment was that it's lacking functionality compared to SPSS for our needs. Thereby not said it's not enough for you.

Wrong is evil and must be defeated. - Jeff Ello[^]

Matlab, Maple, Mathematica, Mathcad, and Scilab are just a few.

http://en.wikipedia.org/wiki/List_of_statistical_packages[^]

Web - BM - RSS - Math - LinkedIn

Cool... How did I not see that... :P

Thx

There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell

Well you had six other threads to read first before you came to mine. Big Grin | :-D

Web - BM - RSS - Math - LinkedIn

I'm trying to keep the conversation going and to vote up on the stuff that is adding more knowledge.

There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell

Statisticians use R, claim it's complete, then hand it to me to "code" into a system. Except ... R is documented, specifically, to not be able to be run programmatically. My R based systems are the buggiest I have.

Oh, and just imagine trying to install all of the required libraries through a firewall! One at a time! UGH. Just tell me the functions and I can write them, sheesh.

BTW, on topic, the Closest thing I use is Aforge.NET.

Need custom software developed? I do custom programming based primarily on MS tools with an emphasis on C# development and consulting.

"And they, since they Were not the one dead, turned to their affairs" -- Robert Frost

"All users always want Excel" --Ennis Lynch

I've never heard about this Aforge.net.. Just had a quick look at it and it sounds like a nice option

Is its focus on images or is there more flexibility to it?

Thank you

There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell

Would Rcpp or RInside help in this situation?

R is scripting language but mostly it is open source library of statistical tests with charting capabilities.It is easy to learn and use. Few lines of "code" (mostly calling library functions) will produce desired answer from simple descriptive stats to complex machine learning tests, signal processing etc. There are more than 4000 packages developed by individuals or universities with documentation. R is easiest to start. Python is slightly more complex and has extensive libraries as well. As with all statistics what really counts is to know which test applies to given problem and what to do with the test results. Both R and Python have lots of free information on the web and extensive literature in bookstores (Amazon).

Cool, it is interesting to see someone talking about R as being the easier one to start with any studies. I might have to look more into R to check that...
I agree with the knowlwdge thing, I'm already enrolled for a statistics course to see if I can get a better grasp on how/why to do the test.

Thanks!

There are no secrets to success. It is the result of preparation, hard work, and learning from failure. Colin Powell

R doesn't do any stats stuff but there are well tested standard packages (libraries) that do. I've used some R in my job and it's fairly easy to work with, and for any vaguely standard stats the packages will just do it for you.

Welcome to the Lounge