<ul class="anchorlinks"><li><a href="#nlp">Natural language processing</a></li><li><a href="#vs">R vs Python for data science: Digging into the differences</a></li><li><a href="#libraries">Libraries for NLP</a></li><li><a href="#data">Data exploration in R and Python</a></li></ul>



In Talking Data, we delve into the rapidly evolving worlds of Natural Language Processing and Generation. Text data is proliferating at a staggering rate, and only advanced coding languages like Python and R will be able to pull insights out of these datasets at scale.



One of the most-asked questions from aspiring data scientists is: “What is the best language for data science? R or Python?”



People looking into data science languages are usually confused about which language they should learn first: R or Python. Both are extremely useful for an array of data science applications, including Natural Language Processing (NLP). To understand the strengths and weaknesses of each, let’s explore R vs Python for data science by analyzing which language works best with NLP.



<h2 class="wp-block-heading" id="nlp">Natural language processing: Teaching computers human words</h2>



Natural language processing means, as the name implies, teaching computers to process natural human languages (English, Hindi, etc.) and perform analyses. NLP can be used on written text or speech data.



<figure class="wp-block-image size-large"><a href="https://www.sisense.com/get/deep-data-exploration-advanced-analytics-and-insights-using-python-and-r/" target="_blank" rel="noreferrer noopener"><img decoding="async" src="https://cdn.sisense.com/wp-content/uploads/SQL_Python_R_CTA_770X250-770x250.jpg" alt="SQL, Python, and R" class="wp-image-65606"/></a></figure>



For our example, we will use written text for our comparison of R vs Python for data science. We are surrounded by written text every day: emails, SMS messages, webpages, books, and much more. Text data plays a vital role in our day-to-day life, which makes NLP a very important area for data scientists to explore.



<h2 class="wp-block-heading">R vs Python for data science: Digging into the differences</h2>



Python and R are two of the top data science languages. Both are open-source and have large user bases. In the real world, it’s often difficult to choose between R and Python for data science and NLP. Here, we’re going to run through some of the must-know info about each of these versatile languages.



<h3 class="wp-block-heading">R: Analytics powerhouse&nbsp;</h3>



R is a tool built by statisticians mainly for mathematics, statistics, research, and data analysis. It’s quite popular for its visualizations: charts, graphs, pictures, and various plots. These visualizations are useful for helping people <a href="https://www.sisense.com/blog/storytelling-through-data-essential-skills-for-analysts/" target="_blank" rel="noreferrer noopener" aria-label="visualize and understand trends (opens in a new tab)">visualize and understand trends</a>, outliers, and patterns in data.



<h3 class="wp-block-heading">Python: Versatile workhorse</h3>



Python is a general-purpose, robust, versatile language with readable syntax. Python’s readable syntax makes it easy to learn and understand, since it can be read much like a human language. Python also integrates well in a variety of different project environments.



<h2 class="wp-block-heading" id="libraries">Libraries for NLP</h2>



Libraries are collections of modules and functions that programmers can include in programs and projects to accomplish specific tasks. Programmers choose different libraries because they help do a particular task more efficiently. For example, the wordcloud library is used to create a word cloud displaying the most frequently used words from a text dataset. We’ll actually do this later in this article.



<h3 class="wp-block-heading">R libraries</h3>



R boasts more than 10,000 libraries, such as Caret, Dplyr, tidyr, caTools, ggplot2, and many others. These support a wide array of uses, such as data analysis, manipulation, visualizations, and machine learning (ML) modeling. Some of the libraries used for NLP are: tm, tidytext, text2vec, and wordcloud. Again, the library you use will be based on your use case. Check out this page from the <a href="https://cran.r-project.org/view=NaturalLanguageProcessing">R Project</a> for a detailed look at the other <a href="https://www.sisense.com/blog/r-python-101-natural-language-processing/" target="_blank" rel="noreferrer noopener" aria-label="libraries used for NLP (opens in a new tab)">libraries used for NLP</a>.



<h3 class="wp-block-heading">Python libraries</h3>



Python has 200+ standard libraries and nearly infinite third-party libraries. Some standard Python libraries are Pandas, Numpy, Scikit-Learn, SciPy, and Matplotlib. These libraries are used for data collection, analysis, data mining, visualizations, and ML modeling. Libraries used for NLP are: NLTK, gensim, SpaCy, glove, and Scikit-Learn. Every library has its own purpose and benefits. For instance, NLTK is excellent for learning and exploring NLP concepts, but it is not meant for production. SpaCy, meanwhile, is a new NLP library that’s designed to be fast and production-ready.



<h2 class="wp-block-heading" id="data">Data exploration in R and Python</h2>



Data exploration is the initial step in data analysis, yielding visualizations like charts or plots that show human users patterns and trends. For our NLP demo, let’s take a dataset of <a href="https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification">commonly used words from Kaggle</a> and do some<a href="https://www.sisense.com/blog/how-skullcandy-uses-predictive-and-sentiment-analysis-to-understand-customers/" target="_blank" rel="noreferrer noopener" aria-label=" data exploration on it in both data science languages (opens in a new tab)"> data exploration on it in both data science languages</a>.



<h3 class="wp-block-heading">Loading data in both R and Python</h3>



First, let’s load training data in both Python and R and check how much time it takes each language.



R code:



<pre class="wp-block-code"><code>#Load data
start_time &lt;- Sys.time()
train&lt;- read.csv("train.csv")
timediff&lt;-difftime(Sys.time(),start_time)
cat("Time taken to load csv is: ",timediff, units(timediff))</code></pre>



Output:



The time taken to load the csv is 3.038562 minutes.



Python code:



<pre class="wp-block-code"><code># Load Data
start_time = time.time()
tr_data=pd.read_csv("train.csv")
time_diff=time.time()-start_time
print("Time taken to load csv is: {} seconds ".format(time_diff))</code></pre>



Output:&nbsp;&nbsp;



The time taken to load the CSV is 17.733536 seconds.



It takes significantly less time for Python to load the CSV than for R to load the same dataset.



<figure class="wp-block-image size-large"><a href="https://www.sisense.com/get/deep-data-exploration-advanced-analytics-and-insights-using-python-and-r/" target="_blank" rel="noreferrer noopener"><img decoding="async" src="https://cdn.sisense.com/wp-content/uploads/SQL_Python_R_CTA_770X250-770x250.jpg" alt="SQL, Python, and R" class="wp-image-65606"/></a></figure>



<h3 class="wp-block-heading">Create a word cloud to find the most-repeated words</h3>



Word clouds are a representation of the words in a dataset as a cluster of words: The more frequently a word appears in the text data, the bigger and bolder it appears in the word cloud.



Let’s create a word cloud of the 50 most frequent words in our training dataset, using both the Python and R programming languages.



R code:



<pre class="wp-block-code"><code>library(wordcloud)
start_time &lt;- Sys.time()
wordcloud(d_toxic$word,d_toxic$freq,min.freq=100,max.words = 50, colors=brewer.pal(8, "Dark2"))
timediff&lt;-difftime(Sys.time(),start_time)
cat("Time take to create wordcloud is ",timediff, units(timediff))</code></pre>



Output:



The time taken to create the word cloud is 0.2203801 seconds.



<figure class="wp-block-image"><img decoding="async" src="https://lh4.googleusercontent.com/28z9FcxlHf4Pxyss1L848F1W-tfJBgBTc5YV_Xdys_77EfWBFLaJL7CtXrO8mr-2_IFQtwrRXQniTWp922gVEZJShkE1l-1S355OF83btGSK_LJ97rSUXxCLY_xiD56Vtc52t6bA" alt=""/></figure>



Word cloud of top 50 words in R 



Python code:



<pre class="wp-block-code"><code>start_time = time.time()
wordcloud = WordCloud(max_words=50, background_color="white", collocations=False).generate(text)
time_diff = time.time() - start_time
print("Time taken to create wordcloud is {} seconds".format(time_diff) )
plt.imshow(wordcloud, interpolation='bilinear') # display the generated image
plt.title(f"Most popular 50 words")
plt.show()</code></pre>



Output:



The time taken to create the word cloud was 129.392610 seconds.



<figure class="wp-block-image"><img decoding="async" src="https://lh3.googleusercontent.com/-C_Y3UKPw987N8OIvJvW0gUJtC3C_osEKI7PTYp844AVwhXfIlFL1taYaCY022Ewkc2ZZcfergIFJV6eogtn-AXrOI3A8fITwXePAVtZXmXOCKU0m8sDFDm4eE3GGn7bqN1Ca3e3" alt=""/><figcaption> Word cloud of top 50 words in Python </figcaption></figure>



Let’s compare the time taken by both data science languages for data loading and word cloud creation.



<div class="table-responsive"><figure class="wp-block-table table table-bordered"><table><tbody><tr><td> </td><td>R (time taken)</td><td>Python (time taken)</td></tr><tr><td>Data loading</td><td>3.038562 minutes</td><td>17.7335 seconds </td></tr><tr><td>Word cloud</td><td>0.2203801 seconds</td><td>129.3926 seconds ~ 2 minutes</td></tr></tbody></table></figure></div>



We can see that in both programming languages, the most frequent words are similar, but the<a href="https://www.sisense.com/blog/storytelling-through-data-essential-skills-for-analysts/" target="_blank" rel="noreferrer noopener" aria-label=" visualization is more beautiful (opens in a new tab)"> visualization is more beautiful</a> in the R programming language. Additionally, the time taken in word cloud creation is lower in R compared to Python. However, the time taken to load the data in Python is significantly lower compared with R.



<h2 class="wp-block-heading">Modeling in R and Python</h2>



When we say “modeling” in data science, we mean teaching a program to learn from training data using machine learning algorithms. In modeling, we find relationships between input variables and the target variable. There are many machine learning algorithms like KNN, Naive Bayes, SVM, Logistic Regression, Decision Trees, Random Forest, XG-Boost, etc. to find relationships between input and target variables.&nbsp;



For example, a customer has written a review for a product (let’s say pasta), so that review (in text) is our input variable, and classifying that text as a positive review or negative review is our target variable.&nbsp;



Review 1: The pasta is very tasty.



Review 2: The pasta is cheap and not tasty.



So, Review 1 is a positive review and Review 2 is a negative review. ML algorithms learn from the input and output variables of past data and predict the target variable for new data.



Here, we will implement the XG-Boost algorithm, an algorithm that learns on the basis of training data (which we loaded earlier in both R and Python programming languages) with the help of probability and statistics. We’ll create a model that helps <a href="https://www.sisense.com/whitepapers/the-biggest-bi-trends-this-year/" target="_blank" rel="noreferrer noopener" aria-label="predict future trends (opens in a new tab)">predict future trends</a>; the system keeps on learning and makes decisions on the basis of what it has learned.



R code:



<pre class="wp-block-code"><code>ctrain &lt;- xgb.DMatrix(Matrix(data.matrix(X_train)), label = (y_train))
xb &lt;- xgboost(ctrain, max_depth = 100, 
 nround=100, 
 eval_metric = "auc",
 objective = "binary:logistic") 
y_pred=predict(xb, y_val)
val-auc=auc(y_pred,y_val)</code></pre>



Output:&nbsp; val-auc: 0.80443



Python code:



<pre class="wp-block-code"><code>clf = xgb.XGBClassifier(max_depth=100,min_samples_split=100);
clf.fit(X_train, y_train)
train_fpr, train_tpr, thresholds = roc_curve(y_train, clf.predict_proba(X_train)&#91;:,1])
train-auc= auc(train_fpr, train_tpr)
val_fpr, val_tpr, thresholds = roc_curve(y_val, clf.predict_proba(X_val)&#91;:,1])
val-auc= auc(val_fpr, val_tpr)</code></pre>



Output:&nbsp; val-auc: 0.83768 XG-Boost performs similarly in both programming languages.



<h2 class="wp-block-heading">Comparison of Python and R for NLP</h2>



Using XG-Boost to model the text data resulted in an almost identical score for Python and R. There are many performance metrics to evaluate performance of Machine Learning models. Here we have used the AUC score; AUC means “area under the curve.” This metric can be used in classification analyses to identify a model’s ability to predict a desired attribute, based on the training data. The slightly higher AUC we get via Python is how we know it performs slightly better than R.



Nowadays text data is huge, so Deep Learning also comes into the picture. Deep learning works well with Big Data sets, and it is based on the concept of our brain cells (neurons), which is the root of the term “Artificial Neural Networks.” As the amount of data in a dataset increases, many ML algorithms’ performance becomes stagnant, but Deep Learning performance improves with increases in data volume.



<figure class="wp-block-image"><img decoding="async" src="https://lh5.googleusercontent.com/KGvolWqD76j-02pvGvEGAd8FApPZaV5Jx3qR9oLsoGH65aG2GFff7q5i9n2IybuSIwMfOnr8RAGxJDG6DGIVeBOS6Oe6P05Gct5aLBs1YHqAZ2fS_5cOlaqZD4qm95n68svVOLXW" alt=""/></figure>



We can also implement deep learning models like Bidirectional LSTM to improve our performance further. Deep Learning models use Keras and Tensorflow API, which are built in Python. However, an R interface for Keras is now available for programming in R. The Keras R package allows us to enjoy the benefit of R programming while having access to the capabilities of the Python Keras package — a powerful combination.



<h2 class="wp-block-heading">Conclusion</h2>



So, after all that: What’s the best language for data science and NLP: Python or R? The answer is, it depends on your personal preferences and what you’re trying to do in your analysis!



All common and necessary data science tasks (data loading, data analysis, data exploration, data preprocessing, data featurization, data modeling, and predictive modeling) are available in both R programming and Python languages. Both languages are user-friendly and easy to implement. There are also a wide array of libraries available for both languages for text processing, text analysis, and text modeling. All ML and deep learning models can be easily implemented in both languages. Deep learning models in R can actually even run the capabilities of the Python APIs in the back end, meaning there’s an added benefit to being conversant in both languages.



A dedicated data expert never stops developing their skills. While you can certainly start with either language, learning a little bit of the other and building more knowledge over time will definitely help make you more capable in the long run.



Nidhi Bansal is Data Scientist, Machine Learning/Artificial Intelligence enthusiast, and writer who loves to experiment with data and write about it. She has over a decade of experience in software development in various programming languages and holds a B.Tech and M.E in Electronics and Communications Engineering.

R vs Python: What’s the Best Language for Natural Language Processing?

curve-image-unique-image-unique