This is a method for the variable scan for multiple regression models, where fewer variables are checked for maximum prediction. It is not usually analyzed regression usually, the gradual regression can be considered as a type of exctraction method. There are three ways: Selection forward execution backward gradient starts the front selection of zero model (ie the form only on fixed terms), and adds each step X X to the form: First select the X variable, and the absolute value of the association is variable Y is the biggest. Second, add a X variable to the form so that the X is increasing Varity is the older statistical value F. Continue until one step, if the corresponding value value per v changer X in the form is less than a predefined value, the X stopped select the back and forward methods Is the opposite of that perfectly. Starting from the full form (ie, the form that contains all independent variables), and each step removes one variable of the form: Select one of the smaller c statistics compatible with all X variables in the current form, and then remove the Standard stop is: if, in a step Certain, the corresponding value value for each X variable in the current form is larger than the pre-defined threshold value, the rejection process is stopped. The gradual regression is a combination of these two methods: assuming that the front method has two changes enter the current form. The following step is creatively use the front method to determine the variables that did not enter the form after, the X variable is tested in the current form to see Whether each variable is independent in the current form, unnecessary variables are removed from the form (same previous removal) (and why this is not criticized the variables specified in the second step of the front method specified in the first step is less important). Continue to use the front method to determine the X variables that did not enter the form after, then use the background method to test the current form ... turn. Standard stop is: If each variable in the current form is in a certain step, each variable is not in the form is not great. I remember that the progressive return was mentioned in "automated learning" with attaches the status code, and I will take it here. If you are interested, you can check it out online. 5) Ridge regression to prevent the convenience of the form, we often need to add Organization terms when creating a linear form. There is generally L1 organization and L2 regulation. Linear regression is usually released L2 Ridge regression, and the difference between them and the general linear regression is that the term L2 is added to the loss function. The expression of the limited loss function to decline Ridge is as follows: where α is a static coefficient and needs to be adjusted. || \ theta || {2} {2} is the base L2. Reducing Ridge Declaration Laboratories without getting rid of any feature, making the form is relatively stable, but compared to the Lasso downturn, this will make the model more, and the most viable model. The Ridge regression solution is relatively simple, and the less box is generally used. The derivation format is provided using the at least square method here, which is similar to normal linear regression. Let a derived j (θ) equals 0, and get the following formula: The end can get on the end result: Ridge is often used when multiple fonts (self-linked data are significantly linked). Key points * Assumptions of this regression are the same as less slope squares, unless the data set is supposed to be normal; * The value of deflation coefficient, but it will not reach zero, indicating that there is no feature selection feature; * This is a way to organize and use the adjustment L2. Class Sklearn.Linear_Model.ridge (Alpha = 1.0, Fit_intercept = True, Normalize = False, Copy_x = True, Max_Iiter = None, TOL = 0.001, Solver = 'Auto', Random_State = None) 6) Lasso regression usually calls linear regression L1 regression lasso, and is different from the linear regression Because it adds a regular term L1 to the loss function, and the L1 regular term has contained α static coefficient to adjust the average error box of the loss function. The weight of the term and the duration of regularity: The loss of loss is to decline as follows: Lasso slopes can make some smaller features, or even become some of the transactions with a smaller immediate values. Enhance the circular capacity of the model. Lasso regression methods are generally coordinated by descent and slope less angle. The main points * Assumptions of this regression are the same as the micro-boxes, unless the natural situation is not assumed; * Shrink to zero (exactly zero), which helps choose the feature; * This method is organized, the L1 settlement; * If a set of predators is significantly associated, the Lasso picks up only one and reduces the rest to zero. Class Sklearn.Linear_Model.lasso (Alpha = 1.0, Fit_intercept = True, Normalize = False, Precompute = False, Copy_x = True, Max_Iiter = 1000, Tol = 0.0001, Warm_Start = False, Positive = False, Random_State = None, Selection = 'Cyclic') 7) Regression Flex Flexnet is a hybrid model for Lasso and Ridge regression techniques. L1 and L2 training are used training. When there is a useful flexible network when there are many related features, I may choose a single Lasso, you may choose two floppy network. The process feature for settlement between Lasso and Ridge is that they allow for Flex-NET to inherit some stability ridge. Key points * The group effects encourage the severe-link variables; * There is no limit to the number of specific variables; * You may suffer a double shrinkage. Class Sklearn.Linear_Model.ElasticNet (alpha = 1.0, l1_ratio = 0.5, fit_intercept = true, normalize = false, precompute = false, max_iter = 1000, copy_x = true, tol = 0.0001, warm_start = false, positive = false, random_state = none, SELECTION = 'CYCLIC') How to choose the correct regression model? In many kinds of regression models, it is important to target variable dependent, data dimensions and some other basic properties of data to choose the most appropriate technology. The following are the main factors to select the correct regression form: Data Exploring is an essential part of predictive forms. You must be the first step before choosing the correct form, such as the determination of the relationship and the impact of variables; In order to compare different models, we can be analyzing various indicators, such as statistics importance parameter, square R square, Square, AIC, AIC and BIC and error terms. The other is Malukhia Conservative Party. This is mainly achieved from possible deviations in the form by comparing the form in all possible subforms (or carefully selected); Through verifying the best way to assess the form used to predict. Here you can divide the data set into two groups (training group and check-out). Simple mistake gives the average quadratic between the observed value and expected value indicator of predictability; If your data set contains multiple variables, you should not choose how to choose the automatic form, because you do not want to put them in one form at the same time; It will also depend on your goals. It is easier to implement the relatively weak model of the form of high statistical importance; Organization of regression (LSO, Ridge and Elasticnet) in the performance and multi-linear data set changes.