Educationalresearchtechniques.com Website Review

A key concept in machine learning and data science in general is variable selection. Sometimes, a dataset can have hundreds of variables to include in a model. The benefit of variable selection is that it reduces the amount of useless information aka noise in the model. By removing noise it can improve the learning process and help to stabilize the estimates. In this post, we will look at two ways to do this.  These two common approaches are the univariate approach and the greedy approach. The univariate approach selects variables that are most related to the dependent variable based on a metric. The greedy approach will alone remove a variable if getting rid of it does not affect the model’s performance. We will now move to our first example which is the univariate approach using Python. We will use the VietNamH dataset from the pydataset library. Are goal is to predict how much a family spends on medical expenses. Below is the initial code. Are data is called df. If you use the head function, you will see that we need to convert several variables to dummy variables. Below is the We are now ready to actual use the univariate approach. This involves the use of two different clas*es in Python. The SelectPercentile clas* checking a variable’s performance in the context of regression.  Below is the code to run the an*lysis. function to iterate over the data. The output is placed in the print statement. Below is the code and output for this. F-score: 62.42   for feature age  F-score: 33.86   for feature educyr  F-score: 3.17    for feature se*  F-score: 106.35  for feature hhsize  F-score: 14.82   for feature fa*m  F-score: 5.95    for feature urban  You can see the f-score for all of the independent variables. You can decide for yourself which to include. The greedy approach only removes variables if they do not impact model performance. We are using the same dataset so all we have to do is run and scoring metric. Finally, we run the an*lysis and print the results. The code is below with the output.