Mathematica

I’ve been using Wolfram Mathematica a lot lately. From Microeconomics classes to Econometrics to data analysis and demonstrations. Here’s what I make of it, good and bad.

The Good

Picture of an article about Mathematica in 1988

Mathematica in 1988

Mathematica truly is state of the art. It recently turned 25, which is quite some time for any piece of software. It’s been the tool I’ve used most frequently for data analysis when given the chance to choose. It’s powerful, it’s simple and it’s highly productive. If you do scientific work and/or data analysis and haven’t already, I think you should give it a try and decide for yourself.

The Language

Stephen Wolfram himself and the guys at Wolfram Research have spent a lot of time in language design and it shows. Here’s what he says about it:

“I’ve been doing language design now for 35 years—and it’s the hardest intellectual activity I know. It requires a curious mixture of clear thinking, aesthetics and pragmatic judgement. And it involves always seeking the deepest possible understanding, and trying to do the broadest unification—to come up in the end with the cleanest and “most obvious” primitives to represent things.”

The Wolfram Language is a descendant of the Lisp family of programming languages, even though it doesn’t look like it.

I’ve been programming in functional programming languages for quite some time now, and since then I’ve worked with Haskell, Erlang, Common Lisp, Scheme and even a little bit of Clojure. In my opinion, the Wolfram Language is definitely among the top ones. And for the kind of work that I do, the functional programming paradigm fits perfectly. Use the right tool for the right job, right?

The Environment

Besides the programming language, the Notebook environment is surprisingly good. I didn’t think so when I first started using it. It was different, unusual. But after some time I’ve grown used to it, and not only that but I also began missing it when working in something else. I like it better even compared to other notebook style environment like IPython notebook. This is a matter of taste, obviously, so YMMV.

Integration

One of the first things that caught my attention is that Mathematica has a “curated data backend” where you can request data from the Wolfram servers (assuming you’re online) without ever leaving it. Of course it’s far from complete, but it does have some interesting Demographic, Geographic and Economic data¹ I can work with for prototyping or exploratory analysis. It has a lot of other kinds of data as well (more on this later). This is the same data used in Wolfram|Alpha, which also integrates nicely with Mathematica.

While still in its infancy, I really hope they keep improving this service and adding more data over time!

Cross-platform

This is basic and a must-have.

Batteries included

I was quite surprised to see how easy it was to start working with geographic data in Mathematica. It works with Shapefiles (arguably the de facto standard for this kind of data) out of the box. No third-party plugins, add-on, libraries or you-name-it. In fact, it works with quite a lot of other data formats as well, some more well known than others. This includes geographic, image, sound, tabular, XML, CAD and many other kinds of data. There’s a list here.

In this regard, this is a very different user experience than that of Python or R, both of which I also use a lot.

Mathematica 8 logo

Mathematica 8 logo

Of course, being able to read or write the data is nothing if you can’t work with it. There’s a wealth of functions and data for many different kids of use. From Chemistry to Image processing. I’ve tried to explore the Mathematica ecosystem a lot, and I feel like I still have only seen 5% or so. There’s a lot of specialized stuff, so I don’t think that someone would ever benefit of this 100%, of course. Still, it’s very interesting to see how flexible Mathematica can be and how many fields it’s been applied.

Compared to MATLAB, Mathematica packs everything in a single package: Statistics, Curve Fitting, Optimization, Finance, Signal Processing, etc., which makes it a lot cheaper if you need some of these to do work. I don’t have enough experience with MATLAB so I can’t say much with regard to performance or usability, though. The little experience I do have with it left me with a very good impression.

There are many more things to talk about Mathematica, but I chose to write just about the ones I care about the most and use frequently.

The Bad

While I like much of Mathematica, I don’t mean to imply that it is a “silver bullet”. It isn’t. There are some things I dislike about it and things I think Wolfram should do differently.

These things I list here are about how Mathematica is managed, or should be managed, when taking in consideration the strong growth of open source alternatives like R and Python, which are in my opinion the strongest candidates for data analysis in the longer term.

Here’s what Google Trends shows:

The CDF thing

Please Wolfram, just forget about CDF. There was a time when it made sense, but that time has passed. We are in 2015 already. No proprietary plugins on the internet, please! Why not just output to HTML5, Javascript, SVG instead?

I would love to use Mathematica-created content on my blog, but I can’t because my readers will not have the CDF player installed, and neither would I want them to install it.

The community around Python (here) and R (here and here) are already exploring this, and so should Wolfram.

Package Management/Installation

Talking about communities, Mathematica already has a good community around it. It even has a StackExchange site, Mathematica.SE. But I was surprised to learn that there isn’t a central repository for third-party packages. Yes, I know there is the Wolfram Library Archive where I can download (sometimes decades old) packages, but I’m not talking about that. I’m talking about an automated system, something similar to Python’s pip, and R’s CRAN. Even .NET has one already, for crying out loud! (See NuGet.)

Licensing

Mathematica licensing and pricing is actually transparent. You have a bunch of options, you look for the one that fits your use case and you can go and buy directly from their website. You just need a credit card. Compare that to other vendors where you have to send an e-mail, wait for a “personalized” (aka expensive) quote and talk to a local sales representative. Quick, right?

Anyway, I don’t know if Wolfram has ever considered opening Mathematica up (as in FOSS), but if it hasn’t, it should. Maybe they can find a sustainable business model around it, maybe they can’t, but that doesn’t mean they shouldn’t look into it.

I would say that if Mathematica were open source it could compete head on with R, and maybe Julia wouldn’t need to exist.

But if Wolfram can’t make it open source, they should make it free for individuals, or at the very least free to students. Something along the lines of what SAS has done with SAS University.

Think this is too unlikely? Well, Wolfram has already done something quite similar recently with Raspberry Pi. Yup, you can use Mathematica for free with Raspberry Pi. To me, this suggests that Wolfram is indeed looking to expand its market share, mainly with regard to the so called Internet of Things, where it’ll probably be competing with (guess who?) Python.

I think they got things right this time. Maybe they can expand on that line of thought.

Pin It

Windows Internals

MCTS Microsoft Certified Technology Specialist Windows Internals Logo

Microsoft Windows Internals Specialist

Não costumo fazer alarde em relação à certificações, mas acredito que essa vale um blog post. :)

Nesta semana realizei a prova de certificação Windows Internals e, com muito gosto, posso dizer que fui aprovado!

Quando soube a respeito dessa prova, ainda em 2011, fiquei bastante interessado. Segue a descrição do exame:

“This exam validates deep technical skills in the area of Windows Internals. Including troubleshooting operating systems that are not performing as expected or applications that are not working correctly, identifying code defects, and developing and debugging applications that run unmanaged code or that are tightly integrated with the operating system, such as Microsoft SQL Server, third party applications, antivirus software, and device drivers.”

Há alguns meses lançaram no Defrag Tools do Channel 9 um episódio falando do exame, e decidi que realmente era hora de tentar.

Esse exame marcou um fim de um ciclo para mim. Hoje estou me dedicando bastante a outras áreas de TI, relacionados ou não diretamente a Windows, mas tenho certeza que o conhecimento adquirido e a certificação ainda vão me auxiliar muito ao longo da minha carreira, além de dar mais pique para continuar gravando os meus treinamentos de Windows Internals pela Sr. Nimbus.

Os skills mensurados durante o exame foram basicamente:

  • Identifying Architectural Components (16%)
  • Designing Solutions (15%)
  • Monitoring Windows (14%)
  • Analyzing User Mode (18%)
  • Analyzing Kernel Mode (19%)
  • Debugging Windows (18%)

Para informações à respeito do que cai em cada tópico, esse post do blog MSDN NtDebugging lista o material e ferramentas que foram cobrados, e no episódio do Defrag Tools já mencionado acima algumas dessas ferramentas são demonstradas.

Pontos gerais da prova

A prova em si é bem prática, com muito debugging tanto em user mode quanto kernel mode. Cobrou bastante comandos do Windbg, e alguns parâmetros. Também caíram questões de desenvolvimento de soluções – drivers e apps Win32 – com direito a código-fonte em C e perguntas sobre parâmetros de APIs em algumas questões. Outro ponto que caiu bastante foi troubleshooting de drivers e aplicativos mal comportados com ferramentas Sysinternals e do WDK.

No geral, eu gostei da prova. Bastante prática e objetiva. As questões e cenários eram bem mais simples do que encontramos no dia a dia mas o suficiente, na minha opinião, para aferir se o profissional realmente conhece os conceitos do sistema operacional, kernel e API Win32, e se tem experiência real realizando trobleshooting, tuning e desenvolvimento nativo e baixo-nível.

EOL da versão 2008

O lado ruim, entretanto, é que a prova não estará mais disponível. Foi descontinuada no dia 31/07, um dia após o meu exame.

Infelizmente a Microsoft ainda não disponibilizou uma versão atualizada da prova para as novas versões do sistema operacional. Levando em conta que os livros Windows Internals do Server 2008 R2 (Client 7) acabaram de ser lançados e que os de 2012 ainda nem têm previsão, não acredito que teremos outra prova tão cedo, o que é uma pena.

O pessoal do Defrag Tools até tentou intervir junto ao time de certificações da Microsoft para que ela mantivesse o exame disponível por mais algum tempo, mas sem sucesso.

Na minha opinião, deveriam ter mantido essa prova, pelo menos até introduzir a versão atualizada. De qualquer maneira, isso me forçou a finalmente criar coragem e realizá-la depois de enrolar por muito tempo, caso contrário teria que esperar uma próxima versão em um futuro distante…

Que venham os próximos desafios!

Pin It