Posted 2021-04-15Updated 2021-04-152 hours read (About 13601 words)

Titanic data

결정 트리, 랜덤포레스트, XGBoost, lightBGM, CATBoost 비교

전처리

1	!pip install catboost

Requirement already satisfied: catboost in /usr/local/lib/python3.7/dist-packages (0.25.1)
Requirement already satisfied: plotly in /usr/local/lib/python3.7/dist-packages (from catboost) (4.4.1)
Requirement already satisfied: graphviz in /usr/local/lib/python3.7/dist-packages (from catboost) (0.10.1)
Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from catboost) (1.15.0)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from catboost) (3.2.2)
Requirement already satisfied: pandas>=0.24.0 in /usr/local/lib/python3.7/dist-packages (from catboost) (1.1.5)
Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from catboost) (1.4.1)
Requirement already satisfied: numpy>=1.16.0 in /usr/local/lib/python3.7/dist-packages (from catboost) (1.19.5)
Requirement already satisfied: retrying>=1.3.3 in /usr/local/lib/python3.7/dist-packages (from plotly->catboost) (1.3.3)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->catboost) (2.8.1)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->catboost) (1.3.1)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->catboost) (2.4.7)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->catboost) (0.10.0)
Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.24.0->catboost) (2018.9)

import os
import random

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import xgboost as xgb
import lightgbm as lgbm
import catboost as cb
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.metrics import accuracy_score
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.preprocessing import LabelEncoder
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.ensemble import AdaBoostClassifier
from sklearn.neighbors import KNeighborsClassifier

from sklearn import metrics    
from sklearn.model_selection import RandomizedSearchCV

def set_seed(seed_value):
    random.seed(seed_value)
    np.random.seed(seed_value)
    os.environ["PYTHONHASHSEED"] = str(seed_value)
    

SEED = 42
set_seed(SEED)

train_df = pd.read_csv('/content/sample_data/titanic_train.csv')
test_df = pd.read_csv('/content/sample_data/titanic_test.csv')
print(f"Train shape: {train_df.shape}")
train_df.sample(3)

Train shape: (891, 12)

	PassengerId	Survived	Pclass	Name	Sex	Age	SibSp	Parch	Ticket	Fare	Cabin	Embarked
709	710	1	3	Moubarek, Master. Halim Gonios ("William George")	male	NaN	1	1	2661	15.2458	NaN	C
439	440	0	2	Kvillner, Mr. Johan Henrik Johannesson	male	31.0	0	0	C.A. 18723	10.5000	NaN	S
840	841	0	3	Alhomaki, Mr. Ilmari Rudolf	male	20.0	0	0	SOTON/O2 3101287	7.9250	NaN	S

1 2	print(f"Test shape: {test_df.shape}") test_df.sample(3)

Test shape: (418, 11)

	PassengerId	Pclass	Name	Sex	Age	SibSp	Parch	Ticket	Fare	Cabin	Embarked
20	912	1	Rothschild, Mr. Martin	male	55.00	1	0	PC 17603	59.40	NaN	C
338	1230	2	Denbury, Mr. Herbert	male	25.00	0	0	C.A. 31029	31.50	NaN	S
250	1142	2	West, Miss. Barbara J	female	0.92	1	2	C.A. 34651	27.75	NaN	S

full_df = pd.concat(
    [
        train_df.drop(["PassengerId", "Survived"], axis=1), 
        test_df.drop(["PassengerId"], axis=1),
    ]
)
y_train = train_df["Survived"].values

1	full_df.isna().sum()

Pclass         0
Name           0
Sex            0
Age          263
SibSp          0
Parch          0
Ticket         0
Fare           1
Cabin       1014
Embarked       2
dtype: int64

1	full_df = full_df.drop(["Age", "Cabin"], axis=1)

plt.figure(figsize=(10, 5))
plt.subplot(1, 2, 1)
plt.hist(full_df["Fare"], bins=20)
plt.xticks(fontsize=14)
plt.yticks(fontsize=14)
plt.title("Fare distribution", fontsize=16)

plt.subplot(1, 2, 2)
embarked_info = full_df["Embarked"].value_counts()
plt.bar(embarked_info.index, embarked_info.values)
plt.xticks(fontsize=14)
plt.yticks(fontsize=14)
plt.title("Embarked distribution", fontsize=16);

png

1 2	full_df["Embarked"].fillna("S", inplace=True) full_df["Fare"].fillna(full_df["Fare"].mean(), inplace=True)

full_df["Title"] = full_df["Name"].str.extract(" ([A-Za-z]+)\.")
full_df["Title"] = full_df["Title"].replace(["Ms", "Mlle"], "Miss")
full_df["Title"] = full_df["Title"].replace(["Mme", "Countess", "Lady", "Dona"], "Mrs")
full_df["Title"] = full_df["Title"].replace(["Dr", "Major", "Col", "Sir", "Rev", "Jonkheer", "Capt", "Don"], "Mr")
full_df = full_df.drop(["Name"], axis=1)

1
2
3

full_df["Sex"] = full_df["Sex"].map({"male": 1, "female": 0}).astype(int)    
full_df["Embarked"] = full_df["Embarked"].map({"S": 1, "C": 2, "Q": 3}).astype(int)    
full_df['Title'] = full_df['Title'].map({"Mr": 0, "Miss": 1, "Mrs": 2, "Master": 3}).astype(int)

full_df["TicketNumber"] = full_df["Ticket"].str.split()
full_df["TicketNumber"] = full_df["TicketNumber"].str[-1]
full_df["TicketNumber"] = LabelEncoder().fit_transform(full_df["TicketNumber"])
full_df = full_df.drop(["Ticket"], axis=1)

1 2	full_df["FamilySize"] = full_df["SibSp"] + full_df["Parch"] + 1 full_df["IsAlone"] = full_df["FamilySize"].apply(lambda x: 1 if x == 1 else 0)

1	full_df.head()

	Pclass	Sex	SibSp	Fare	Embarked	Title	TicketNumber	FamilySize	IsAlone
0	3	1	1	7.2500	1	0	209	2	0
1	1	0	1	71.2833	2	2	166	2	0
2	3	0	0	7.9250	1	1	466	1	1
3	1	0	1	53.1000	1	2	67	2	0
4	3	1	0	8.0500	1	0	832	1	1

X_train = full_df[:y_train.shape[0]]
X_test = full_df[y_train.shape[0]:]

print(f"Train X shape: {X_train.shape}")
print(f"Train y shape: {y_train.shape}")
print(f"Test X shape: {X_test.shape}")

Train X shape: (891, 10)
Train y shape: (891,)
Test X shape: (418, 10)

one_hot_cols = ["Embarked", "Title"]
for col in one_hot_cols:
    full_df = pd.concat(
        [full_df, pd.get_dummies(full_df[col], prefix=col)], 
        axis=1, 
        join="inner",
    )
full_df = full_df.drop(one_hot_cols, axis=1)

1 2	scaler = StandardScaler() full_df.loc[:] = scaler.fit_transform(full_df)

1	print(full_df)

       Pclass       Sex     SibSp  ...   Title_1   Title_2   Title_3
0    0.841916  0.743497  0.481288  ... -0.502625 -0.425920 -0.221084
1   -1.546098 -1.344995  0.481288  ... -0.502625  2.347858 -0.221084
2    0.841916 -1.344995 -0.479087  ...  1.989556 -0.425920 -0.221084
3   -1.546098 -1.344995  0.481288  ... -0.502625  2.347858 -0.221084
4    0.841916  0.743497 -0.479087  ... -0.502625 -0.425920 -0.221084
..        ...       ...       ...  ...       ...       ...       ...
413  0.841916  0.743497 -0.479087  ... -0.502625 -0.425920 -0.221084
414 -1.546098 -1.344995 -0.479087  ... -0.502625  2.347858 -0.221084
415  0.841916  0.743497 -0.479087  ... -0.502625 -0.425920 -0.221084
416  0.841916  0.743497 -0.479087  ... -0.502625 -0.425920 -0.221084
417  0.841916  0.743497  0.481288  ... -0.502625 -0.425920  4.523164

[1309 rows x 15 columns]

X_train_norm = full_df[:y_train.shape[0]]
X_test_norm = full_df[y_train.shape[0]:]

print(f"Train norm X shape: {X_train_norm.shape}")
print(f"Train y shape: {y_train.shape}")
print(f"Test norm X shape: {X_test_norm.shape}")

Train norm X shape: (891, 15)
Train y shape: (891,)
Test norm X shape: (418, 15)

1	categorical_columns = ['Sex', 'Embarked', 'Title', 'TicketNumber', 'IsAlone']

1	cross_valid_scores = {}

1	X1_train, X1_test, y1_train, y1_test = train_test_split(X_train, y_train, test_size=0.3)

결정트리 생성

GridSearch

%%time
parameters = {
    "max_depth": [3, 5, 7, 9, 11, 13],
}

model_desicion_tree = DecisionTreeClassifier(
    random_state=SEED,
    class_weight='balanced',
)

model_desicion_tree = GridSearchCV(
    model_desicion_tree, 
    parameters, 
    cv=5,
    scoring='accuracy',
)

model_desicion_tree.fit(X_train, y_train)

print('-----')
print(f'Best parameters {model_desicion_tree.best_params_}')
print(
    f'Mean cross-validated accuracy score of the best_estimator: ' + \
    f'{model_desicion_tree.best_score_:.3f}'
)
cross_valid_scores['desicion_tree'] = model_desicion_tree.best_score_
print('-----')

-----
Best parameters {'max_depth': 11}
Mean cross-validated accuracy score of the best_estimator: 0.817
-----
CPU times: user 191 ms, sys: 4.34 ms, total: 196 ms
Wall time: 205 ms

랜덤 서치

%%time

params = {
    "max_depth":[3, 5, 7, 9, 11, 13],
}

model_desicion_tree_rs = DecisionTreeClassifier(
    random_state=SEED,
    class_weight='balanced',
)

model_desicion_tree_rs = RandomizedSearchCV(model_desicion_tree_rs,params,cv=5,n_iter=50,random_state=0,scoring="accuracy")

model_desicion_tree_rs.fit(X_train, y_train)

print('-----')
print(f'Best parameters {model_desicion_tree_rs.best_params_}')
print(
    f'Mean cross-validated accuracy score of the best_estimator: ' + \
    f'{model_desicion_tree_rs.best_score_:.3f}'
)
cross_valid_scores['desicion_tree'] = model_desicion_tree_rs.best_score_
print('-----')

-----
Best parameters {'max_depth': 11}
Mean cross-validated accuracy score of the best_estimator: 0.817
-----
CPU times: user 166 ms, sys: 680 µs, total: 167 ms
Wall time: 168 ms


/usr/local/lib/python3.7/dist-packages/sklearn/model_selection/_search.py:281: UserWarning: The total space of parameters 6 is smaller than n_iter=50. Running 6 iterations. For exhaustive searches, use GridSearchCV.
  % (grid_size, self.n_iter, grid_size), UserWarning)

하이퍼파라미터 튜닝 전

%%time
model_dtree1=DecisionTreeClassifier(max_depth=5)
model_dtree1.fit(X_train, y_train)
y_pred_dtree1=model_dtree1.predict(X1_test)
print("\n정확도: ", metrics.accuracy_score(y1_test, y_pred_dtree1))
print("-----")

정확도:  0.8582089552238806
-----
CPU times: user 9.36 ms, sys: 46 µs, total: 9.41 ms
Wall time: 9.77 ms

하이퍼 파라미터 튜닝 후

%%time
model_dtree2=DecisionTreeClassifier(max_depth=11)
model_dtree2.fit(X_train, y_train)
y_pred_dtree2=model_dtree2.predict(X1_test)
print("\n정확도: ", metrics.accuracy_score(y1_test, y_pred_dtree2))
print("-----")

정확도:  0.9850746268656716
-----
CPU times: user 9.64 ms, sys: 91 µs, total: 9.73 ms
Wall time: 13.7 ms

랜덤 포레스트

그리드서치

%%time
parameters = {
    "n_estimators": [5, 10, 15, 20, 25], 
    "max_depth": [3, 5, 7, 9, 11, 13],
}

model_random_forest = RandomForestClassifier(
    random_state=SEED,
    class_weight='balanced',
)

model_random_forest = GridSearchCV(
    model_random_forest, 
    parameters, 
    cv=5,
    scoring='accuracy',
)

model_random_forest.fit(X_train, y_train)

print('-----')
print(f'Best parameters {model_random_forest.best_params_}')
print(
    f'Mean cross-validated accuracy score of the best_estimator: '+ \
    f'{model_random_forest.best_score_:.3f}'
)
cross_valid_scores['random_forest'] = model_random_forest.best_score_
print('-----')

-----
Best parameters {'max_depth': 11, 'n_estimators': 25}
Mean cross-validated accuracy score of the best_estimator: 0.844
-----
CPU times: user 4.93 s, sys: 30.4 ms, total: 4.96 s
Wall time: 4.98 s

랜덤 서치

%%time

parameters = {
    "n_estimators": [5, 10, 15, 20, 25], 
    "max_depth": [3, 5, 7, 9, 11, 13],
}
model2_random_forest_rs = RandomForestClassifier(
    random_state=SEED,
    class_weight='balanced',
)

model2_random_forest_rs = RandomizedSearchCV(model2_random_forest_rs,parameters,cv=5,n_iter=50,random_state=0,scoring="accuracy")
model2_random_forest_rs.fit(X_train, y_train)

print('-----')
print(f'Best parameters {model2_random_forest_rs.best_params_}')
print(
    f'Mean cross-validated accuracy score of the best_estimator: '+ \
    f'{model2_random_forest_rs.best_score_:.3f}'
)
cross_valid_scores['random_forest'] = model2_random_forest_rs.best_score_
print('-----')

/usr/local/lib/python3.7/dist-packages/sklearn/model_selection/_search.py:281: UserWarning: The total space of parameters 30 is smaller than n_iter=50. Running 30 iterations. For exhaustive searches, use GridSearchCV.
  % (grid_size, self.n_iter, grid_size), UserWarning)


-----
Best parameters {'n_estimators': 25, 'max_depth': 11}
Mean cross-validated accuracy score of the best_estimator: 0.844
-----
CPU times: user 4.88 s, sys: 35.5 ms, total: 4.92 s
Wall time: 4.92 s

###파라미터 튜닝을 하지않은 randomForest

%%time
model_rf1=RandomForestClassifier(max_depth=5)  #default 값으로 넣으면 과적합 문제때문에 max_depth를 임의로 5로 설정했다.
model_rf1.fit(X_train,y_train)

y_pred_rf1=model_rf1.predict(X1_test)

print('\n정확도 :', metrics.accuracy_score(y1_test, y_pred_rf1))
print("-----")

정확도 : 0.8544776119402985
-----
CPU times: user 187 ms, sys: 1.96 ms, total: 189 ms
Wall time: 190 ms

%%time
model_rf2=RandomForestClassifier(n_estimators= 25, max_depth= 11)
model_rf2.fit(X_train,y_train)

y_pred_rf2=model_rf2.predict(X1_test)

print('\n정확도 :', metrics.accuracy_score(y1_test, y_pred_rf2))
print("-----")

정확도 : 0.9589552238805971
-----
CPU times: user 62.9 ms, sys: 2.11 ms, total: 65 ms
Wall time: 65.6 ms

1	print(model_rf1)

RandomForestClassifier(bootstrap=True, ccp_alpha=0.0, class_weight=None,
                       criterion='gini', max_depth=5, max_features='auto',
                       max_leaf_nodes=None, max_samples=None,
                       min_impurity_decrease=0.0, min_impurity_split=None,
                       min_samples_leaf=1, min_samples_split=2,
                       min_weight_fraction_leaf=0.0, n_estimators=100,
                       n_jobs=None, oob_score=False, random_state=None,
                       verbose=0, warm_start=False)

XGBOOST

gridSearch

%%time
parameters = {
    'max_depth': [3, 5, 7, 9], 
    'n_estimators': [5, 10, 15, 20, 25, 50, 100],
    'learning_rate': [0.01, 0.05, 0.1]
}

model_xgb = xgb.XGBClassifier(
    random_state=SEED,
)

model_xgb = GridSearchCV(
    model_xgb, 
    parameters, 
    cv=5,
    scoring='accuracy',
)

model_xgb.fit(X_train, y_train)

print('-----')
print(f'Best parameters {model_xgb.best_params_}')
print(
    f'Mean cross-validated accuracy score of the best_estimator: ' + 
    f'{model_xgb.best_score_:.3f}'
)
cross_valid_scores['xgboost'] = model_xgb.best_score_
print('-----')

-----
Best parameters {'learning_rate': 0.1, 'max_depth': 7, 'n_estimators': 100}
Mean cross-validated accuracy score of the best_estimator: 0.846
-----
CPU times: user 13.8 s, sys: 189 ms, total: 14 s
Wall time: 14.1 s

xgboost에서 하이퍼파라미터튜닝을 위해 GridSearch를 진행.
time : 14.3 s
Best parameters {‘n_estimators’: 100, ‘max_depth’: 7, ‘learning_rate’: 0.1}

랜덤 서치

%%time
params = {
    'max_depth': [3, 5, 7, 9], 
    'n_estimators': [5, 10, 15, 20, 25, 50, 100],
    'learning_rate': [0.01, 0.05, 0.1]
}
model_xgb_random = xgb.XGBClassifier(
    random_state=SEED,
)
model_xgb_random =RandomizedSearchCV(model_xgb_random ,params,cv=5,n_iter=50,random_state=0,scoring="accuracy")

model_xgb_random .fit(X_train, y_train)

print('-----')
print(f'Best parameters {model_xgb_random .best_params_}')
print(
    f'Mean cross-validated accuracy score of the best_estimator: ' + 
    f'{model_xgb_random .best_score_:.3f}'
)
cross_valid_scores['xgboost'] = model_xgb_random .best_score_
print('-----')

-----
Best parameters {'n_estimators': 100, 'max_depth': 7, 'learning_rate': 0.1}
Mean cross-validated accuracy score of the best_estimator: 0.846
-----
CPU times: user 9.31 s, sys: 113 ms, total: 9.42 s
Wall time: 9.41 s

xgboost에서 두 서치의 성능을 보기위해 똑같은 환경에서 RandomSearch를 진행.
time : 9.46 s
Best parameters {‘n_estimators’: 100, ‘max_depth’: 7, ‘learning_rate’: 0.1}

파라미터튜닝을 하지않은 xgboost

%%time
model_1=xgb.XGBClassifier()
model_1.fit(X_train,y_train)
pred_y1=model_1.predict(X1_test)

print('\n정확도 :', metrics.accuracy_score(y1_test, pred_y1))
print("-----")

정확도 : 0.8843283582089553
-----
CPU times: user 63.7 ms, sys: 993 µs, total: 64.7 ms
Wall time: 63.4 ms

하이퍼 파라미터 적용

%%time
model_2=xgb.XGBClassifier(learning_rate= 0.1, max_depth= 7, n_estimators=100)
model_2.fit(X_train,y_train)

pred_y2=model_2.predict(X1_test)

print('\n정확도 :', metrics.accuracy_score(y1_test, pred_y2))
print("-----")

정확도 : 0.9514925373134329
-----
CPU times: user 118 ms, sys: 2.04 ms, total: 120 ms
Wall time: 119 ms

그리드 서치보다 랜덤 서치의 속도가 더 빠른 것을 알 수있다.
또한 하이퍼 파라미터를 튜닝 한 후의 정확도가 훨씬 올라갔음을 알 수 있다.

lightBGM

GridSearch

%%time
parameters = {
    'n_estimators': [5, 10, 15, 20, 25, 50, 100],
    'learning_rate': [0.01, 0.05, 0.1],
    'num_leaves': [7, 15,  31],
}

model_lgbm = lgbm.LGBMClassifier(
    random_state=SEED,
    class_weight='balanced',
)

model_lgbm = GridSearchCV(
    model_lgbm, 
    parameters, 
    cv=5,
    scoring='accuracy',
)

model_lgbm.fit(
    X_train, 
    y_train, 
    categorical_feature=categorical_columns
)

print('-----')
print(f'Best parameters {model_lgbm.best_params_}')
print(
    f'Mean cross-validated accuracy score of the best_estimator: ' + 
    f'{model_lgbm.best_score_:.3f}'
)
cross_valid_scores['lightgbm'] = model_lgbm.best_score_
print('-----')

/usr/local/lib/python3.7/dist-packages/lightgbm/basic.py:1209: UserWarning: categorical_feature in Dataset is overridden.
New categorical_feature is ['Embarked', 'IsAlone', 'Sex', 'TicketNumber', 'Title']
  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))


-----
Best parameters {'learning_rate': 0.1, 'n_estimators': 25, 'num_leaves': 15}
Mean cross-validated accuracy score of the best_estimator: 0.827
-----
CPU times: user 5.83 s, sys: 346 ms, total: 6.18 s
Wall time: 6.2 s

랜덤 서치

%%time
params = {
    'n_estimators': [5, 10, 15, 20, 25, 50, 100],
    'learning_rate': [0.01, 0.05, 0.1],
    'num_leaves': [7, 15, 31],
}
model_lgbm_random = lgbm.LGBMClassifier(
    random_state=SEED,
)
model_lgbm_random =RandomizedSearchCV(model_lgbm_random ,params,cv=5,n_iter=50,random_state=0,scoring="accuracy")

model_lgbm_random .fit(X_train, y_train)

print('-----')
print(f'Best parameters {model_lgbm_random .best_params_}')
print(
    f'Mean cross-validated accuracy score of the best_estimator: ' + 
    f'{model_lgbm_random .best_score_:.3f}'
)
cross_valid_scores['LightGBM'] = model_lgbm_random .best_score_
print('-----')

-----
Best parameters {'num_leaves': 31, 'n_estimators': 100, 'learning_rate': 0.05}
Mean cross-validated accuracy score of the best_estimator: 0.846
-----
CPU times: user 4.66 s, sys: 210 ms, total: 4.87 s
Wall time: 4.87 s

파라미터튜닝을 하지않은 LightGBM

%%time
model_1=lgbm.LGBMClassifier()
model_1.fit(X_train,y_train)
pred_y1=model_1.predict(X1_test)

print('\n정확도 :', metrics.accuracy_score(y1_test, pred_y1))
print('-----')

정확도 : 0.9514925373134329
-----
CPU times: user 69.5 ms, sys: 3.96 ms, total: 73.5 ms
Wall time: 76.1 ms

하이퍼 파라미터 적용

%%time

model_2=lgbm.LGBMClassifier(num_leaves= 15,n_estimators=25, learning_rate= 0.1)
model_2.fit(X_train,y_train)
pred_y2=model_2.predict(X1_test)

print('\n정확도 :', metrics.accuracy_score(y1_test, pred_y2))
print('-----')

정확도 : 0.8582089552238806
-----
CPU times: user 20.3 ms, sys: 1.01 ms, total: 21.4 ms
Wall time: 22.4 ms

하이퍼 파라미터 적용2

%%time

model_2=lgbm.LGBMClassifier(num_leaves= 31,n_estimators=100, learning_rate= 0.05)
model_2.fit(X_train,y_train)
pred_y2=model_2.predict(X1_test)

print('\n정확도 :', metrics.accuracy_score(y1_test, pred_y2))

정확도 : 0.914179104477612
CPU times: user 66.4 ms, sys: 6.96 ms, total: 73.3 ms
Wall time: 74.2 ms

Catboost

그리드서치

%%time
parameters = {
    'iterations': [5, 10, 15, 20, 25, 50, 100],
    'learning_rate': [0.01, 0.05, 0.1],
    'depth': [3, 5, 7, 9, 11, 13],
}

model_catboost = cb.CatBoostClassifier(
    verbose=False,
)

model_catboost = GridSearchCV(
    model_catboost, 
    parameters, 
    cv=5,
    scoring='accuracy',
)

model_catboost.fit(X_train, y_train)

print('-----')
print(f'Best parameters {model_catboost.best_params_}')
print(
    f'Mean cross-validated accuracy score of the best_estimator: ' + 
    f'{model_catboost.best_score_:.3f}'
)
cross_valid_scores['catboost'] = model_catboost.best_score_
print('-----')

-----
Best parameters {'depth': 13, 'iterations': 100, 'learning_rate': 0.1}
Mean cross-validated accuracy score of the best_estimator: 0.838
-----
CPU times: user 3min 49s, sys: 6.43 s, total: 3min 55s
Wall time: 2min 27s

랜덤 서치

%%time

model_catboost_rs = cb.CatBoostClassifier(
    verbose=False,
)


model_catboost_rs=RandomizedSearchCV(model_catboost_rs ,params,cv=5,n_iter=50,random_state=0,scoring="accuracy")
model_catboost_rs.fit(X_train,y_train)


print('-----')
print(f'Best parameters {model_catboost_rs.best_params_}')
print(
    f'Mean cross-validated accuracy score of the best_estimator: ' + 
    f'{model_catboost_rs.best_score_:.3f}'
)
cross_valid_scores['catboost'] = model_catboost_rs.best_score_
print('-----')

/usr/local/lib/python3.7/dist-packages/sklearn/model_selection/_validation.py:536: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan. Details: 
_catboost.CatBoostError: catboost/private/libs/options/catboost_options.cpp:893: max_leaves option works only with lossguide tree growing

  FitFailedWarning)


-----
Best parameters {'num_leaves': 31, 'n_estimators': 100, 'learning_rate': 0.01}
Mean cross-validated accuracy score of the best_estimator: 0.832
-----
CPU times: user 6.45 s, sys: 702 ms, total: 7.15 s
Wall time: 7.18 s

하이퍼 파라미터 튜닝 전

1	from catboost import Pool, CatBoostClassifier, cv

%%time
model_cb1=cb.CatBoostClassifier()
model_cb1.fit(X_train, y_train)
y_pred_cb1=model_cb1.predict(X1_test)
print("\n정확도: ", metrics.accuracy_score(y1_test, y_pred_cb1))

Learning rate set to 0.009807
0:    learn: 0.6864423    total: 1.15ms    remaining: 1.15s
1:    learn: 0.6796678    total: 3.78ms    remaining: 1.89s
2:    learn: 0.6736018    total: 7.16ms    remaining: 2.38s
3:    learn: 0.6665873    total: 10.6ms    remaining: 2.63s
4:    learn: 0.6600996    total: 12.4ms    remaining: 2.47s
5:    learn: 0.6534897    total: 14.2ms    remaining: 2.36s
6:    learn: 0.6472195    total: 15.8ms    remaining: 2.25s
7:    learn: 0.6415114    total: 17.4ms    remaining: 2.16s
8:    learn: 0.6353287    total: 19ms    remaining: 2.09s
9:    learn: 0.6295821    total: 20.5ms    remaining: 2.03s
10:    learn: 0.6242876    total: 21.6ms    remaining: 1.95s
11:    learn: 0.6184887    total: 23.1ms    remaining: 1.9s
12:    learn: 0.6131840    total: 24.7ms    remaining: 1.87s
13:    learn: 0.6078112    total: 26.2ms    remaining: 1.84s
14:    learn: 0.6030343    total: 27.9ms    remaining: 1.83s
15:    learn: 0.5977374    total: 29.4ms    remaining: 1.81s
16:    learn: 0.5928195    total: 31ms    remaining: 1.79s
17:    learn: 0.5892999    total: 32.1ms    remaining: 1.75s
18:    learn: 0.5855036    total: 33.5ms    remaining: 1.73s
19:    learn: 0.5812113    total: 35.3ms    remaining: 1.73s
20:    learn: 0.5765712    total: 36.8ms    remaining: 1.72s
21:    learn: 0.5720403    total: 38.3ms    remaining: 1.7s
22:    learn: 0.5680552    total: 39.8ms    remaining: 1.69s
23:    learn: 0.5638984    total: 41.4ms    remaining: 1.68s
24:    learn: 0.5596681    total: 42.9ms    remaining: 1.67s
25:    learn: 0.5553698    total: 44.4ms    remaining: 1.66s
26:    learn: 0.5516134    total: 46ms    remaining: 1.66s
27:    learn: 0.5481987    total: 47.5ms    remaining: 1.65s
28:    learn: 0.5447553    total: 49.1ms    remaining: 1.64s
29:    learn: 0.5408223    total: 50.6ms    remaining: 1.64s
30:    learn: 0.5369519    total: 52.1ms    remaining: 1.63s
31:    learn: 0.5330715    total: 53.6ms    remaining: 1.62s
32:    learn: 0.5299491    total: 55.1ms    remaining: 1.61s
33:    learn: 0.5267190    total: 56.4ms    remaining: 1.6s
34:    learn: 0.5231781    total: 57.9ms    remaining: 1.6s
35:    learn: 0.5208511    total: 58.8ms    remaining: 1.57s
36:    learn: 0.5177569    total: 60.3ms    remaining: 1.57s
37:    learn: 0.5150180    total: 61.8ms    remaining: 1.56s
38:    learn: 0.5118389    total: 63.4ms    remaining: 1.56s
39:    learn: 0.5092100    total: 64.9ms    remaining: 1.56s
40:    learn: 0.5064001    total: 66.3ms    remaining: 1.55s
41:    learn: 0.5038346    total: 67.9ms    remaining: 1.55s
42:    learn: 0.5013958    total: 69.5ms    remaining: 1.54s
43:    learn: 0.4984976    total: 71ms    remaining: 1.54s
44:    learn: 0.4968538    total: 72.5ms    remaining: 1.54s
45:    learn: 0.4948834    total: 73.6ms    remaining: 1.53s
46:    learn: 0.4922797    total: 75ms    remaining: 1.52s
47:    learn: 0.4898559    total: 76.5ms    remaining: 1.52s
48:    learn: 0.4882097    total: 78.1ms    remaining: 1.51s
49:    learn: 0.4858639    total: 79.7ms    remaining: 1.51s
50:    learn: 0.4839866    total: 81.4ms    remaining: 1.51s
51:    learn: 0.4815496    total: 83ms    remaining: 1.51s
52:    learn: 0.4792559    total: 84.6ms    remaining: 1.51s
53:    learn: 0.4770495    total: 86.2ms    remaining: 1.51s
54:    learn: 0.4747478    total: 87.8ms    remaining: 1.51s
55:    learn: 0.4727874    total: 89.3ms    remaining: 1.5s
56:    learn: 0.4712635    total: 90.4ms    remaining: 1.5s
57:    learn: 0.4694542    total: 92ms    remaining: 1.49s
58:    learn: 0.4679098    total: 93.5ms    remaining: 1.49s
59:    learn: 0.4666425    total: 94.6ms    remaining: 1.48s
60:    learn: 0.4651955    total: 96.2ms    remaining: 1.48s
61:    learn: 0.4629838    total: 97.8ms    remaining: 1.48s
62:    learn: 0.4613419    total: 99.3ms    remaining: 1.48s
63:    learn: 0.4596180    total: 101ms    remaining: 1.48s
64:    learn: 0.4582795    total: 103ms    remaining: 1.48s
65:    learn: 0.4567869    total: 104ms    remaining: 1.48s
66:    learn: 0.4554270    total: 106ms    remaining: 1.48s
67:    learn: 0.4537690    total: 108ms    remaining: 1.48s
68:    learn: 0.4520165    total: 109ms    remaining: 1.47s
69:    learn: 0.4504949    total: 111ms    remaining: 1.47s
70:    learn: 0.4492505    total: 113ms    remaining: 1.47s
71:    learn: 0.4477988    total: 115ms    remaining: 1.48s
72:    learn: 0.4470907    total: 116ms    remaining: 1.47s
73:    learn: 0.4456455    total: 117ms    remaining: 1.47s
74:    learn: 0.4439957    total: 119ms    remaining: 1.46s
75:    learn: 0.4424007    total: 120ms    remaining: 1.46s
76:    learn: 0.4411398    total: 122ms    remaining: 1.46s
77:    learn: 0.4393015    total: 123ms    remaining: 1.46s
78:    learn: 0.4377323    total: 125ms    remaining: 1.46s
79:    learn: 0.4366939    total: 127ms    remaining: 1.46s
80:    learn: 0.4357767    total: 128ms    remaining: 1.45s
81:    learn: 0.4342050    total: 129ms    remaining: 1.45s
82:    learn: 0.4329284    total: 131ms    remaining: 1.45s
83:    learn: 0.4319752    total: 134ms    remaining: 1.46s
84:    learn: 0.4305898    total: 136ms    remaining: 1.46s
85:    learn: 0.4292832    total: 140ms    remaining: 1.49s
86:    learn: 0.4279310    total: 143ms    remaining: 1.5s
87:    learn: 0.4265082    total: 145ms    remaining: 1.5s
88:    learn: 0.4252939    total: 147ms    remaining: 1.5s
89:    learn: 0.4243060    total: 148ms    remaining: 1.5s
90:    learn: 0.4236286    total: 150ms    remaining: 1.5s
91:    learn: 0.4225358    total: 151ms    remaining: 1.49s
92:    learn: 0.4220645    total: 153ms    remaining: 1.49s
93:    learn: 0.4210174    total: 154ms    remaining: 1.49s
94:    learn: 0.4199543    total: 156ms    remaining: 1.48s
95:    learn: 0.4188798    total: 158ms    remaining: 1.48s
96:    learn: 0.4179159    total: 159ms    remaining: 1.48s
97:    learn: 0.4168335    total: 161ms    remaining: 1.48s
98:    learn: 0.4164371    total: 162ms    remaining: 1.47s
99:    learn: 0.4156060    total: 163ms    remaining: 1.47s
100:    learn: 0.4147372    total: 165ms    remaining: 1.47s
101:    learn: 0.4137972    total: 167ms    remaining: 1.47s
102:    learn: 0.4133928    total: 168ms    remaining: 1.46s
103:    learn: 0.4123880    total: 169ms    remaining: 1.46s
104:    learn: 0.4114707    total: 171ms    remaining: 1.46s
105:    learn: 0.4106357    total: 173ms    remaining: 1.46s
106:    learn: 0.4098909    total: 174ms    remaining: 1.45s
107:    learn: 0.4091430    total: 176ms    remaining: 1.45s
108:    learn: 0.4085852    total: 177ms    remaining: 1.45s
109:    learn: 0.4079336    total: 181ms    remaining: 1.46s
110:    learn: 0.4068446    total: 184ms    remaining: 1.47s
111:    learn: 0.4059413    total: 186ms    remaining: 1.47s
112:    learn: 0.4051074    total: 187ms    remaining: 1.47s
113:    learn: 0.4045525    total: 189ms    remaining: 1.47s
114:    learn: 0.4039265    total: 190ms    remaining: 1.46s
115:    learn: 0.4033559    total: 192ms    remaining: 1.46s
116:    learn: 0.4026959    total: 194ms    remaining: 1.46s
117:    learn: 0.4021791    total: 195ms    remaining: 1.46s
118:    learn: 0.4013881    total: 197ms    remaining: 1.46s
119:    learn: 0.4005294    total: 199ms    remaining: 1.46s
120:    learn: 0.4000457    total: 201ms    remaining: 1.46s
121:    learn: 0.3993558    total: 203ms    remaining: 1.46s
122:    learn: 0.3987516    total: 205ms    remaining: 1.46s
123:    learn: 0.3983254    total: 206ms    remaining: 1.46s
124:    learn: 0.3981335    total: 207ms    remaining: 1.45s
125:    learn: 0.3975284    total: 209ms    remaining: 1.45s
126:    learn: 0.3968148    total: 211ms    remaining: 1.45s
127:    learn: 0.3962797    total: 213ms    remaining: 1.45s
128:    learn: 0.3955650    total: 214ms    remaining: 1.45s
129:    learn: 0.3949156    total: 216ms    remaining: 1.45s
130:    learn: 0.3943753    total: 218ms    remaining: 1.44s
131:    learn: 0.3939230    total: 219ms    remaining: 1.44s
132:    learn: 0.3933117    total: 221ms    remaining: 1.44s
133:    learn: 0.3928159    total: 222ms    remaining: 1.44s
134:    learn: 0.3922164    total: 224ms    remaining: 1.44s
135:    learn: 0.3918342    total: 225ms    remaining: 1.43s
136:    learn: 0.3912796    total: 227ms    remaining: 1.43s
137:    learn: 0.3905823    total: 229ms    remaining: 1.43s
138:    learn: 0.3898593    total: 230ms    remaining: 1.43s
139:    learn: 0.3893237    total: 232ms    remaining: 1.43s
140:    learn: 0.3888304    total: 234ms    remaining: 1.43s
141:    learn: 0.3884924    total: 236ms    remaining: 1.42s
142:    learn: 0.3881707    total: 238ms    remaining: 1.42s
143:    learn: 0.3878014    total: 239ms    remaining: 1.42s
144:    learn: 0.3873020    total: 241ms    remaining: 1.42s
145:    learn: 0.3868475    total: 243ms    remaining: 1.42s
146:    learn: 0.3862259    total: 245ms    remaining: 1.42s
147:    learn: 0.3858858    total: 246ms    remaining: 1.42s
148:    learn: 0.3856409    total: 248ms    remaining: 1.42s
149:    learn: 0.3855390    total: 249ms    remaining: 1.41s
150:    learn: 0.3850505    total: 251ms    remaining: 1.41s
151:    learn: 0.3844543    total: 253ms    remaining: 1.41s
152:    learn: 0.3839649    total: 254ms    remaining: 1.41s
153:    learn: 0.3832490    total: 256ms    remaining: 1.41s
154:    learn: 0.3828330    total: 257ms    remaining: 1.4s
155:    learn: 0.3823213    total: 259ms    remaining: 1.4s
156:    learn: 0.3819428    total: 261ms    remaining: 1.4s
157:    learn: 0.3813389    total: 262ms    remaining: 1.4s
158:    learn: 0.3809790    total: 264ms    remaining: 1.4s
159:    learn: 0.3805731    total: 266ms    remaining: 1.39s
160:    learn: 0.3799352    total: 267ms    remaining: 1.39s
161:    learn: 0.3794951    total: 269ms    remaining: 1.39s
162:    learn: 0.3790710    total: 271ms    remaining: 1.39s
163:    learn: 0.3786079    total: 272ms    remaining: 1.39s
164:    learn: 0.3783021    total: 274ms    remaining: 1.39s
165:    learn: 0.3778044    total: 275ms    remaining: 1.38s
166:    learn: 0.3773915    total: 277ms    remaining: 1.38s
167:    learn: 0.3772473    total: 278ms    remaining: 1.38s
168:    learn: 0.3768942    total: 280ms    remaining: 1.37s
169:    learn: 0.3762634    total: 281ms    remaining: 1.37s
170:    learn: 0.3758273    total: 283ms    remaining: 1.37s
171:    learn: 0.3754783    total: 284ms    remaining: 1.37s
172:    learn: 0.3753974    total: 285ms    remaining: 1.36s
173:    learn: 0.3752110    total: 287ms    remaining: 1.36s
174:    learn: 0.3750740    total: 288ms    remaining: 1.36s
175:    learn: 0.3747491    total: 290ms    remaining: 1.35s
176:    learn: 0.3740551    total: 291ms    remaining: 1.35s
177:    learn: 0.3736829    total: 293ms    remaining: 1.35s
178:    learn: 0.3733106    total: 295ms    remaining: 1.35s
179:    learn: 0.3730030    total: 296ms    remaining: 1.35s
180:    learn: 0.3727365    total: 298ms    remaining: 1.35s
181:    learn: 0.3726708    total: 299ms    remaining: 1.34s
182:    learn: 0.3724766    total: 300ms    remaining: 1.34s
183:    learn: 0.3720786    total: 302ms    remaining: 1.34s
184:    learn: 0.3717549    total: 304ms    remaining: 1.34s
185:    learn: 0.3713830    total: 305ms    remaining: 1.33s
186:    learn: 0.3710702    total: 307ms    remaining: 1.33s
187:    learn: 0.3707764    total: 308ms    remaining: 1.33s
188:    learn: 0.3707388    total: 309ms    remaining: 1.33s
189:    learn: 0.3704293    total: 311ms    remaining: 1.32s
190:    learn: 0.3700587    total: 313ms    remaining: 1.32s
191:    learn: 0.3699942    total: 314ms    remaining: 1.32s
192:    learn: 0.3696027    total: 318ms    remaining: 1.33s
193:    learn: 0.3695621    total: 319ms    remaining: 1.32s
194:    learn: 0.3691724    total: 320ms    remaining: 1.32s
195:    learn: 0.3688717    total: 322ms    remaining: 1.32s
196:    learn: 0.3685092    total: 323ms    remaining: 1.32s
197:    learn: 0.3682595    total: 325ms    remaining: 1.32s
198:    learn: 0.3679573    total: 327ms    remaining: 1.32s
199:    learn: 0.3678256    total: 328ms    remaining: 1.31s
200:    learn: 0.3675041    total: 331ms    remaining: 1.31s
201:    learn: 0.3670877    total: 332ms    remaining: 1.31s
202:    learn: 0.3668744    total: 334ms    remaining: 1.31s
203:    learn: 0.3665374    total: 337ms    remaining: 1.31s
204:    learn: 0.3662455    total: 338ms    remaining: 1.31s
205:    learn: 0.3662061    total: 339ms    remaining: 1.31s
206:    learn: 0.3659549    total: 341ms    remaining: 1.3s
207:    learn: 0.3657520    total: 342ms    remaining: 1.3s
208:    learn: 0.3651990    total: 344ms    remaining: 1.3s
209:    learn: 0.3650540    total: 346ms    remaining: 1.3s
210:    learn: 0.3648195    total: 347ms    remaining: 1.3s
211:    learn: 0.3643852    total: 349ms    remaining: 1.29s
212:    learn: 0.3643304    total: 350ms    remaining: 1.29s
213:    learn: 0.3639860    total: 352ms    remaining: 1.29s
214:    learn: 0.3636619    total: 356ms    remaining: 1.3s
215:    learn: 0.3634521    total: 359ms    remaining: 1.3s
216:    learn: 0.3631182    total: 361ms    remaining: 1.3s
217:    learn: 0.3629000    total: 362ms    remaining: 1.3s
218:    learn: 0.3628546    total: 363ms    remaining: 1.29s
219:    learn: 0.3626871    total: 364ms    remaining: 1.29s
220:    learn: 0.3625245    total: 365ms    remaining: 1.29s
221:    learn: 0.3622324    total: 367ms    remaining: 1.28s
222:    learn: 0.3620531    total: 368ms    remaining: 1.28s
223:    learn: 0.3617895    total: 369ms    remaining: 1.28s
224:    learn: 0.3613159    total: 371ms    remaining: 1.28s
225:    learn: 0.3611309    total: 372ms    remaining: 1.27s
226:    learn: 0.3608805    total: 373ms    remaining: 1.27s
227:    learn: 0.3603855    total: 374ms    remaining: 1.27s
228:    learn: 0.3603646    total: 375ms    remaining: 1.26s
229:    learn: 0.3601897    total: 376ms    remaining: 1.26s
230:    learn: 0.3601518    total: 377ms    remaining: 1.25s
231:    learn: 0.3601231    total: 377ms    remaining: 1.25s
232:    learn: 0.3599573    total: 379ms    remaining: 1.25s
233:    learn: 0.3599279    total: 379ms    remaining: 1.24s
234:    learn: 0.3595377    total: 381ms    remaining: 1.24s
235:    learn: 0.3592815    total: 382ms    remaining: 1.24s
236:    learn: 0.3589589    total: 383ms    remaining: 1.23s
237:    learn: 0.3586604    total: 385ms    remaining: 1.23s
238:    learn: 0.3584376    total: 386ms    remaining: 1.23s
239:    learn: 0.3583077    total: 387ms    remaining: 1.23s
240:    learn: 0.3580581    total: 388ms    remaining: 1.22s
241:    learn: 0.3577347    total: 390ms    remaining: 1.22s
242:    learn: 0.3577167    total: 390ms    remaining: 1.22s
243:    learn: 0.3576102    total: 391ms    remaining: 1.21s
244:    learn: 0.3573276    total: 393ms    remaining: 1.21s
245:    learn: 0.3570597    total: 394ms    remaining: 1.21s
246:    learn: 0.3570164    total: 395ms    remaining: 1.2s
247:    learn: 0.3568129    total: 396ms    remaining: 1.2s
248:    learn: 0.3566178    total: 397ms    remaining: 1.2s
249:    learn: 0.3563533    total: 399ms    remaining: 1.2s
250:    learn: 0.3560881    total: 400ms    remaining: 1.19s
251:    learn: 0.3559918    total: 401ms    remaining: 1.19s
252:    learn: 0.3556106    total: 403ms    remaining: 1.19s
253:    learn: 0.3552943    total: 404ms    remaining: 1.19s
254:    learn: 0.3551120    total: 405ms    remaining: 1.18s
255:    learn: 0.3549299    total: 407ms    remaining: 1.18s
256:    learn: 0.3546934    total: 408ms    remaining: 1.18s
257:    learn: 0.3544919    total: 410ms    remaining: 1.18s
258:    learn: 0.3543437    total: 411ms    remaining: 1.18s
259:    learn: 0.3541476    total: 413ms    remaining: 1.17s
260:    learn: 0.3538909    total: 414ms    remaining: 1.17s
261:    learn: 0.3537310    total: 416ms    remaining: 1.17s
262:    learn: 0.3535340    total: 417ms    remaining: 1.17s
263:    learn: 0.3534128    total: 419ms    remaining: 1.17s
264:    learn: 0.3531355    total: 420ms    remaining: 1.17s
265:    learn: 0.3529686    total: 422ms    remaining: 1.16s
266:    learn: 0.3527602    total: 423ms    remaining: 1.16s
267:    learn: 0.3524416    total: 425ms    remaining: 1.16s
268:    learn: 0.3520992    total: 426ms    remaining: 1.16s
269:    learn: 0.3519173    total: 428ms    remaining: 1.16s
270:    learn: 0.3517764    total: 429ms    remaining: 1.16s
271:    learn: 0.3516484    total: 431ms    remaining: 1.15s
272:    learn: 0.3513071    total: 432ms    remaining: 1.15s
273:    learn: 0.3511635    total: 434ms    remaining: 1.15s
274:    learn: 0.3507677    total: 435ms    remaining: 1.15s
275:    learn: 0.3504876    total: 437ms    remaining: 1.15s
276:    learn: 0.3503400    total: 438ms    remaining: 1.14s
277:    learn: 0.3502119    total: 440ms    remaining: 1.14s
278:    learn: 0.3500942    total: 442ms    remaining: 1.14s
279:    learn: 0.3500215    total: 443ms    remaining: 1.14s
280:    learn: 0.3498679    total: 445ms    remaining: 1.14s
281:    learn: 0.3494740    total: 446ms    remaining: 1.14s
282:    learn: 0.3493393    total: 448ms    remaining: 1.13s
283:    learn: 0.3492231    total: 452ms    remaining: 1.14s
284:    learn: 0.3489882    total: 453ms    remaining: 1.14s
285:    learn: 0.3488344    total: 454ms    remaining: 1.13s
286:    learn: 0.3486306    total: 456ms    remaining: 1.13s
287:    learn: 0.3484715    total: 457ms    remaining: 1.13s
288:    learn: 0.3483100    total: 458ms    remaining: 1.13s
289:    learn: 0.3481646    total: 460ms    remaining: 1.13s
290:    learn: 0.3480016    total: 461ms    remaining: 1.12s
291:    learn: 0.3477597    total: 463ms    remaining: 1.12s
292:    learn: 0.3476850    total: 464ms    remaining: 1.12s
293:    learn: 0.3476563    total: 466ms    remaining: 1.12s
294:    learn: 0.3474599    total: 467ms    remaining: 1.12s
295:    learn: 0.3471187    total: 469ms    remaining: 1.11s
296:    learn: 0.3469055    total: 470ms    remaining: 1.11s
297:    learn: 0.3466185    total: 472ms    remaining: 1.11s
298:    learn: 0.3465162    total: 473ms    remaining: 1.11s
299:    learn: 0.3464710    total: 475ms    remaining: 1.11s
300:    learn: 0.3463116    total: 476ms    remaining: 1.11s
301:    learn: 0.3462097    total: 478ms    remaining: 1.1s
302:    learn: 0.3460624    total: 479ms    remaining: 1.1s
303:    learn: 0.3458503    total: 481ms    remaining: 1.1s
304:    learn: 0.3457330    total: 483ms    remaining: 1.1s
305:    learn: 0.3454716    total: 484ms    remaining: 1.1s
306:    learn: 0.3453798    total: 490ms    remaining: 1.1s
307:    learn: 0.3452549    total: 494ms    remaining: 1.11s
308:    learn: 0.3451684    total: 497ms    remaining: 1.11s
309:    learn: 0.3448519    total: 498ms    remaining: 1.11s
310:    learn: 0.3447327    total: 500ms    remaining: 1.11s
311:    learn: 0.3445245    total: 502ms    remaining: 1.11s
312:    learn: 0.3444054    total: 503ms    remaining: 1.1s
313:    learn: 0.3442425    total: 505ms    remaining: 1.1s
314:    learn: 0.3441252    total: 506ms    remaining: 1.1s
315:    learn: 0.3438394    total: 508ms    remaining: 1.1s
316:    learn: 0.3437785    total: 510ms    remaining: 1.1s
317:    learn: 0.3436580    total: 511ms    remaining: 1.1s
318:    learn: 0.3434121    total: 513ms    remaining: 1.09s
319:    learn: 0.3434015    total: 514ms    remaining: 1.09s
320:    learn: 0.3432309    total: 515ms    remaining: 1.09s
321:    learn: 0.3430105    total: 517ms    remaining: 1.09s
322:    learn: 0.3430040    total: 518ms    remaining: 1.08s
323:    learn: 0.3428893    total: 519ms    remaining: 1.08s
324:    learn: 0.3427908    total: 521ms    remaining: 1.08s
325:    learn: 0.3426938    total: 522ms    remaining: 1.08s
326:    learn: 0.3424298    total: 526ms    remaining: 1.08s
327:    learn: 0.3422154    total: 528ms    remaining: 1.08s
328:    learn: 0.3420228    total: 531ms    remaining: 1.08s
329:    learn: 0.3419105    total: 533ms    remaining: 1.08s
330:    learn: 0.3417604    total: 535ms    remaining: 1.08s
331:    learn: 0.3417005    total: 537ms    remaining: 1.08s
332:    learn: 0.3415153    total: 538ms    remaining: 1.08s
333:    learn: 0.3413515    total: 540ms    remaining: 1.08s
334:    learn: 0.3412281    total: 542ms    remaining: 1.07s
335:    learn: 0.3412011    total: 543ms    remaining: 1.07s
336:    learn: 0.3410229    total: 544ms    remaining: 1.07s
337:    learn: 0.3409003    total: 546ms    remaining: 1.07s
338:    learn: 0.3407465    total: 547ms    remaining: 1.07s
339:    learn: 0.3405374    total: 549ms    remaining: 1.06s
340:    learn: 0.3404344    total: 550ms    remaining: 1.06s
341:    learn: 0.3403684    total: 552ms    remaining: 1.06s
342:    learn: 0.3400907    total: 554ms    remaining: 1.06s
343:    learn: 0.3398546    total: 555ms    remaining: 1.06s
344:    learn: 0.3397414    total: 557ms    remaining: 1.06s
345:    learn: 0.3396128    total: 558ms    remaining: 1.05s
346:    learn: 0.3395383    total: 560ms    remaining: 1.05s
347:    learn: 0.3393384    total: 561ms    remaining: 1.05s
348:    learn: 0.3391067    total: 564ms    remaining: 1.05s
349:    learn: 0.3389667    total: 567ms    remaining: 1.05s
350:    learn: 0.3387429    total: 568ms    remaining: 1.05s
351:    learn: 0.3385726    total: 571ms    remaining: 1.05s
352:    learn: 0.3384251    total: 573ms    remaining: 1.05s
353:    learn: 0.3381414    total: 575ms    remaining: 1.05s
354:    learn: 0.3379741    total: 576ms    remaining: 1.05s
355:    learn: 0.3378329    total: 577ms    remaining: 1.04s
356:    learn: 0.3377206    total: 578ms    remaining: 1.04s
357:    learn: 0.3375257    total: 579ms    remaining: 1.04s
358:    learn: 0.3373769    total: 581ms    remaining: 1.04s
359:    learn: 0.3372383    total: 582ms    remaining: 1.03s
360:    learn: 0.3370866    total: 583ms    remaining: 1.03s
361:    learn: 0.3370745    total: 584ms    remaining: 1.03s
362:    learn: 0.3369338    total: 586ms    remaining: 1.03s
363:    learn: 0.3368527    total: 587ms    remaining: 1.02s
364:    learn: 0.3367434    total: 588ms    remaining: 1.02s
365:    learn: 0.3365576    total: 589ms    remaining: 1.02s
366:    learn: 0.3364554    total: 591ms    remaining: 1.02s
367:    learn: 0.3363495    total: 592ms    remaining: 1.02s
368:    learn: 0.3361014    total: 593ms    remaining: 1.01s
369:    learn: 0.3359145    total: 595ms    remaining: 1.01s
370:    learn: 0.3358289    total: 596ms    remaining: 1.01s
371:    learn: 0.3356082    total: 598ms    remaining: 1.01s
372:    learn: 0.3354244    total: 599ms    remaining: 1.01s
373:    learn: 0.3351816    total: 601ms    remaining: 1s
374:    learn: 0.3349759    total: 602ms    remaining: 1s
375:    learn: 0.3348936    total: 604ms    remaining: 1s
376:    learn: 0.3346268    total: 605ms    remaining: 1s
377:    learn: 0.3343676    total: 607ms    remaining: 999ms
378:    learn: 0.3341921    total: 609ms    remaining: 998ms
379:    learn: 0.3339870    total: 611ms    remaining: 996ms
380:    learn: 0.3337607    total: 612ms    remaining: 995ms
381:    learn: 0.3337276    total: 614ms    remaining: 993ms
382:    learn: 0.3335497    total: 615ms    remaining: 991ms
383:    learn: 0.3333378    total: 617ms    remaining: 990ms
384:    learn: 0.3331444    total: 619ms    remaining: 989ms
385:    learn: 0.3328706    total: 620ms    remaining: 987ms
386:    learn: 0.3327889    total: 622ms    remaining: 985ms
387:    learn: 0.3326313    total: 624ms    remaining: 985ms
388:    learn: 0.3324981    total: 626ms    remaining: 983ms
389:    learn: 0.3323998    total: 628ms    remaining: 982ms
390:    learn: 0.3323033    total: 629ms    remaining: 980ms
391:    learn: 0.3322477    total: 631ms    remaining: 978ms
392:    learn: 0.3319772    total: 632ms    remaining: 977ms
393:    learn: 0.3317993    total: 634ms    remaining: 975ms
394:    learn: 0.3316346    total: 636ms    remaining: 973ms
395:    learn: 0.3315460    total: 637ms    remaining: 972ms
396:    learn: 0.3314495    total: 639ms    remaining: 970ms
397:    learn: 0.3313958    total: 640ms    remaining: 969ms
398:    learn: 0.3313636    total: 642ms    remaining: 966ms
399:    learn: 0.3312036    total: 643ms    remaining: 965ms
400:    learn: 0.3310957    total: 645ms    remaining: 964ms
401:    learn: 0.3309663    total: 647ms    remaining: 962ms
402:    learn: 0.3309633    total: 648ms    remaining: 959ms
403:    learn: 0.3308602    total: 649ms    remaining: 958ms
404:    learn: 0.3307102    total: 651ms    remaining: 957ms
405:    learn: 0.3306083    total: 653ms    remaining: 955ms
406:    learn: 0.3303933    total: 655ms    remaining: 954ms
407:    learn: 0.3303254    total: 657ms    remaining: 953ms
408:    learn: 0.3301241    total: 658ms    remaining: 951ms
409:    learn: 0.3300132    total: 660ms    remaining: 950ms
410:    learn: 0.3298714    total: 662ms    remaining: 949ms
411:    learn: 0.3297545    total: 666ms    remaining: 951ms
412:    learn: 0.3296896    total: 670ms    remaining: 953ms
413:    learn: 0.3293730    total: 672ms    remaining: 951ms
414:    learn: 0.3291582    total: 673ms    remaining: 949ms
415:    learn: 0.3289779    total: 675ms    remaining: 947ms
416:    learn: 0.3288515    total: 676ms    remaining: 945ms
417:    learn: 0.3286225    total: 677ms    remaining: 943ms
418:    learn: 0.3284417    total: 679ms    remaining: 941ms
419:    learn: 0.3282645    total: 680ms    remaining: 939ms
420:    learn: 0.3282057    total: 682ms    remaining: 937ms
421:    learn: 0.3280742    total: 683ms    remaining: 936ms
422:    learn: 0.3279897    total: 684ms    remaining: 933ms
423:    learn: 0.3279414    total: 685ms    remaining: 931ms
424:    learn: 0.3278043    total: 687ms    remaining: 929ms
425:    learn: 0.3277831    total: 687ms    remaining: 926ms
426:    learn: 0.3276888    total: 689ms    remaining: 924ms
427:    learn: 0.3276084    total: 690ms    remaining: 922ms
428:    learn: 0.3274960    total: 692ms    remaining: 921ms
429:    learn: 0.3273183    total: 693ms    remaining: 919ms
430:    learn: 0.3271913    total: 694ms    remaining: 917ms
431:    learn: 0.3271639    total: 696ms    remaining: 915ms
432:    learn: 0.3269948    total: 698ms    remaining: 913ms
433:    learn: 0.3268786    total: 700ms    remaining: 912ms
434:    learn: 0.3267243    total: 702ms    remaining: 911ms
435:    learn: 0.3266135    total: 703ms    remaining: 910ms
436:    learn: 0.3264746    total: 705ms    remaining: 908ms
437:    learn: 0.3260983    total: 706ms    remaining: 906ms
438:    learn: 0.3260188    total: 708ms    remaining: 904ms
439:    learn: 0.3259822    total: 709ms    remaining: 902ms
440:    learn: 0.3257988    total: 710ms    remaining: 901ms
441:    learn: 0.3256852    total: 712ms    remaining: 899ms
442:    learn: 0.3255033    total: 713ms    remaining: 897ms
443:    learn: 0.3254455    total: 715ms    remaining: 895ms
444:    learn: 0.3253134    total: 716ms    remaining: 893ms
445:    learn: 0.3252633    total: 717ms    remaining: 891ms
446:    learn: 0.3251332    total: 719ms    remaining: 889ms
447:    learn: 0.3249648    total: 720ms    remaining: 887ms
448:    learn: 0.3248574    total: 722ms    remaining: 886ms
449:    learn: 0.3247845    total: 723ms    remaining: 884ms
450:    learn: 0.3245354    total: 724ms    remaining: 882ms
451:    learn: 0.3244206    total: 726ms    remaining: 880ms
452:    learn: 0.3243711    total: 727ms    remaining: 878ms
453:    learn: 0.3243288    total: 729ms    remaining: 877ms
454:    learn: 0.3242242    total: 730ms    remaining: 875ms
455:    learn: 0.3239490    total: 732ms    remaining: 873ms
456:    learn: 0.3238705    total: 733ms    remaining: 871ms
457:    learn: 0.3238606    total: 734ms    remaining: 869ms
458:    learn: 0.3236528    total: 735ms    remaining: 867ms
459:    learn: 0.3235039    total: 737ms    remaining: 865ms
460:    learn: 0.3234211    total: 738ms    remaining: 863ms
461:    learn: 0.3233266    total: 740ms    remaining: 861ms
462:    learn: 0.3231303    total: 741ms    remaining: 859ms
463:    learn: 0.3229646    total: 742ms    remaining: 857ms
464:    learn: 0.3227962    total: 744ms    remaining: 855ms
465:    learn: 0.3226196    total: 745ms    remaining: 854ms
466:    learn: 0.3223571    total: 746ms    remaining: 852ms
467:    learn: 0.3222110    total: 748ms    remaining: 850ms
468:    learn: 0.3220662    total: 749ms    remaining: 848ms
469:    learn: 0.3219840    total: 750ms    remaining: 846ms
470:    learn: 0.3219059    total: 752ms    remaining: 844ms
471:    learn: 0.3218827    total: 754ms    remaining: 843ms
472:    learn: 0.3217921    total: 755ms    remaining: 841ms
473:    learn: 0.3216864    total: 757ms    remaining: 840ms
474:    learn: 0.3216762    total: 758ms    remaining: 837ms
475:    learn: 0.3215663    total: 759ms    remaining: 836ms
476:    learn: 0.3214919    total: 760ms    remaining: 834ms
477:    learn: 0.3213385    total: 762ms    remaining: 832ms
478:    learn: 0.3212131    total: 764ms    remaining: 830ms
479:    learn: 0.3211449    total: 765ms    remaining: 829ms
480:    learn: 0.3210588    total: 767ms    remaining: 827ms
481:    learn: 0.3210299    total: 768ms    remaining: 825ms
482:    learn: 0.3207860    total: 770ms    remaining: 824ms
483:    learn: 0.3206680    total: 773ms    remaining: 825ms
484:    learn: 0.3204886    total: 776ms    remaining: 823ms
485:    learn: 0.3201907    total: 778ms    remaining: 823ms
486:    learn: 0.3201631    total: 780ms    remaining: 821ms
487:    learn: 0.3200827    total: 782ms    remaining: 820ms
488:    learn: 0.3199985    total: 784ms    remaining: 819ms
489:    learn: 0.3198093    total: 786ms    remaining: 818ms
490:    learn: 0.3197330    total: 787ms    remaining: 816ms
491:    learn: 0.3195681    total: 789ms    remaining: 814ms
492:    learn: 0.3194094    total: 790ms    remaining: 813ms
493:    learn: 0.3193594    total: 792ms    remaining: 811ms
494:    learn: 0.3192246    total: 793ms    remaining: 809ms
495:    learn: 0.3190883    total: 795ms    remaining: 808ms
496:    learn: 0.3190022    total: 796ms    remaining: 806ms
497:    learn: 0.3189700    total: 798ms    remaining: 804ms
498:    learn: 0.3187582    total: 799ms    remaining: 803ms
499:    learn: 0.3186219    total: 801ms    remaining: 801ms
500:    learn: 0.3185476    total: 803ms    remaining: 799ms
501:    learn: 0.3184275    total: 804ms    remaining: 798ms
502:    learn: 0.3183448    total: 806ms    remaining: 796ms
503:    learn: 0.3182237    total: 808ms    remaining: 795ms
504:    learn: 0.3179514    total: 809ms    remaining: 793ms
505:    learn: 0.3179136    total: 811ms    remaining: 792ms
506:    learn: 0.3176711    total: 813ms    remaining: 790ms
507:    learn: 0.3175794    total: 814ms    remaining: 788ms
508:    learn: 0.3174719    total: 816ms    remaining: 787ms
509:    learn: 0.3172911    total: 817ms    remaining: 785ms
510:    learn: 0.3172861    total: 818ms    remaining: 783ms
511:    learn: 0.3172683    total: 819ms    remaining: 781ms
512:    learn: 0.3172510    total: 820ms    remaining: 779ms
513:    learn: 0.3171707    total: 822ms    remaining: 777ms
514:    learn: 0.3170907    total: 824ms    remaining: 776ms
515:    learn: 0.3170215    total: 825ms    remaining: 774ms
516:    learn: 0.3169492    total: 827ms    remaining: 772ms
517:    learn: 0.3168907    total: 828ms    remaining: 771ms
518:    learn: 0.3166368    total: 830ms    remaining: 769ms
519:    learn: 0.3165108    total: 831ms    remaining: 767ms
520:    learn: 0.3164277    total: 834ms    remaining: 767ms
521:    learn: 0.3163670    total: 836ms    remaining: 766ms
522:    learn: 0.3162119    total: 838ms    remaining: 764ms
523:    learn: 0.3159909    total: 840ms    remaining: 763ms
524:    learn: 0.3158627    total: 843ms    remaining: 763ms
525:    learn: 0.3157602    total: 845ms    remaining: 762ms
526:    learn: 0.3157078    total: 847ms    remaining: 760ms
527:    learn: 0.3156260    total: 848ms    remaining: 758ms
528:    learn: 0.3154974    total: 850ms    remaining: 756ms
529:    learn: 0.3153165    total: 851ms    remaining: 755ms
530:    learn: 0.3151412    total: 853ms    remaining: 753ms
531:    learn: 0.3150721    total: 854ms    remaining: 751ms
532:    learn: 0.3149426    total: 856ms    remaining: 750ms
533:    learn: 0.3148334    total: 858ms    remaining: 748ms
534:    learn: 0.3146896    total: 859ms    remaining: 747ms
535:    learn: 0.3144815    total: 861ms    remaining: 745ms
536:    learn: 0.3143912    total: 862ms    remaining: 744ms
537:    learn: 0.3143369    total: 864ms    remaining: 742ms
538:    learn: 0.3142618    total: 868ms    remaining: 743ms
539:    learn: 0.3141539    total: 870ms    remaining: 741ms
540:    learn: 0.3140318    total: 872ms    remaining: 740ms
541:    learn: 0.3139340    total: 874ms    remaining: 739ms
542:    learn: 0.3138693    total: 883ms    remaining: 743ms
543:    learn: 0.3137137    total: 884ms    remaining: 741ms
544:    learn: 0.3136775    total: 888ms    remaining: 741ms
545:    learn: 0.3135136    total: 890ms    remaining: 740ms
546:    learn: 0.3132667    total: 892ms    remaining: 738ms
547:    learn: 0.3131847    total: 897ms    remaining: 740ms
548:    learn: 0.3131351    total: 899ms    remaining: 738ms
549:    learn: 0.3130413    total: 900ms    remaining: 736ms
550:    learn: 0.3129377    total: 901ms    remaining: 735ms
551:    learn: 0.3127922    total: 903ms    remaining: 733ms
552:    learn: 0.3126559    total: 904ms    remaining: 731ms
553:    learn: 0.3125413    total: 906ms    remaining: 729ms
554:    learn: 0.3123807    total: 907ms    remaining: 727ms
555:    learn: 0.3122795    total: 908ms    remaining: 725ms
556:    learn: 0.3121609    total: 910ms    remaining: 724ms
557:    learn: 0.3120492    total: 911ms    remaining: 722ms
558:    learn: 0.3120059    total: 912ms    remaining: 720ms
559:    learn: 0.3118993    total: 914ms    remaining: 718ms
560:    learn: 0.3117650    total: 915ms    remaining: 716ms
561:    learn: 0.3116605    total: 917ms    remaining: 714ms
562:    learn: 0.3114923    total: 918ms    remaining: 713ms
563:    learn: 0.3114171    total: 919ms    remaining: 711ms
564:    learn: 0.3112964    total: 921ms    remaining: 709ms
565:    learn: 0.3111950    total: 922ms    remaining: 707ms
566:    learn: 0.3111227    total: 924ms    remaining: 705ms
567:    learn: 0.3109404    total: 925ms    remaining: 703ms
568:    learn: 0.3109345    total: 926ms    remaining: 701ms
569:    learn: 0.3107864    total: 927ms    remaining: 699ms
570:    learn: 0.3107062    total: 928ms    remaining: 697ms
571:    learn: 0.3105907    total: 930ms    remaining: 696ms
572:    learn: 0.3104622    total: 932ms    remaining: 694ms
573:    learn: 0.3104295    total: 933ms    remaining: 693ms
574:    learn: 0.3102594    total: 935ms    remaining: 691ms
575:    learn: 0.3099801    total: 936ms    remaining: 689ms
576:    learn: 0.3098861    total: 938ms    remaining: 688ms
577:    learn: 0.3097799    total: 940ms    remaining: 686ms
578:    learn: 0.3096591    total: 941ms    remaining: 684ms
579:    learn: 0.3094861    total: 943ms    remaining: 683ms
580:    learn: 0.3094233    total: 945ms    remaining: 681ms
581:    learn: 0.3093261    total: 947ms    remaining: 680ms
582:    learn: 0.3092554    total: 948ms    remaining: 678ms
583:    learn: 0.3091984    total: 950ms    remaining: 677ms
584:    learn: 0.3090779    total: 951ms    remaining: 675ms
585:    learn: 0.3089114    total: 953ms    remaining: 673ms
586:    learn: 0.3087072    total: 955ms    remaining: 672ms
587:    learn: 0.3086577    total: 957ms    remaining: 671ms
588:    learn: 0.3085933    total: 959ms    remaining: 669ms
589:    learn: 0.3084591    total: 960ms    remaining: 667ms
590:    learn: 0.3082292    total: 962ms    remaining: 666ms
591:    learn: 0.3081197    total: 964ms    remaining: 665ms
592:    learn: 0.3078835    total: 966ms    remaining: 663ms
593:    learn: 0.3077445    total: 968ms    remaining: 661ms
594:    learn: 0.3076817    total: 969ms    remaining: 660ms
595:    learn: 0.3076539    total: 971ms    remaining: 658ms
596:    learn: 0.3075941    total: 973ms    remaining: 657ms
597:    learn: 0.3074667    total: 975ms    remaining: 655ms
598:    learn: 0.3074483    total: 976ms    remaining: 653ms
599:    learn: 0.3073675    total: 977ms    remaining: 651ms
600:    learn: 0.3072547    total: 979ms    remaining: 650ms
601:    learn: 0.3072363    total: 980ms    remaining: 648ms
602:    learn: 0.3070790    total: 982ms    remaining: 646ms
603:    learn: 0.3069212    total: 984ms    remaining: 645ms
604:    learn: 0.3067527    total: 993ms    remaining: 648ms
605:    learn: 0.3066531    total: 995ms    remaining: 647ms
606:    learn: 0.3065907    total: 997ms    remaining: 645ms
607:    learn: 0.3064529    total: 998ms    remaining: 644ms
608:    learn: 0.3063562    total: 999ms    remaining: 642ms
609:    learn: 0.3062009    total: 1s    remaining: 640ms
610:    learn: 0.3060596    total: 1s    remaining: 638ms
611:    learn: 0.3059804    total: 1s    remaining: 636ms
612:    learn: 0.3059462    total: 1s    remaining: 635ms
613:    learn: 0.3058976    total: 1.01s    remaining: 633ms
614:    learn: 0.3057840    total: 1.01s    remaining: 632ms
615:    learn: 0.3057076    total: 1.01s    remaining: 630ms
616:    learn: 0.3055833    total: 1.01s    remaining: 628ms
617:    learn: 0.3055278    total: 1.01s    remaining: 627ms
618:    learn: 0.3054147    total: 1.01s    remaining: 625ms
619:    learn: 0.3052816    total: 1.02s    remaining: 624ms
620:    learn: 0.3052263    total: 1.02s    remaining: 623ms
621:    learn: 0.3051848    total: 1.02s    remaining: 621ms
622:    learn: 0.3051363    total: 1.02s    remaining: 619ms
623:    learn: 0.3049397    total: 1.02s    remaining: 618ms
624:    learn: 0.3049107    total: 1.03s    remaining: 616ms
625:    learn: 0.3047204    total: 1.03s    remaining: 615ms
626:    learn: 0.3045337    total: 1.03s    remaining: 613ms
627:    learn: 0.3044718    total: 1.03s    remaining: 611ms
628:    learn: 0.3044015    total: 1.03s    remaining: 610ms
629:    learn: 0.3043145    total: 1.03s    remaining: 608ms
630:    learn: 0.3042671    total: 1.04s    remaining: 606ms
631:    learn: 0.3041630    total: 1.04s    remaining: 605ms
632:    learn: 0.3040370    total: 1.04s    remaining: 603ms
633:    learn: 0.3039320    total: 1.04s    remaining: 601ms
634:    learn: 0.3038157    total: 1.04s    remaining: 599ms
635:    learn: 0.3037399    total: 1.04s    remaining: 598ms
636:    learn: 0.3037030    total: 1.04s    remaining: 596ms
637:    learn: 0.3035112    total: 1.05s    remaining: 595ms
638:    learn: 0.3033812    total: 1.05s    remaining: 594ms
639:    learn: 0.3032892    total: 1.05s    remaining: 593ms
640:    learn: 0.3031651    total: 1.05s    remaining: 591ms
641:    learn: 0.3030900    total: 1.06s    remaining: 590ms
642:    learn: 0.3030082    total: 1.06s    remaining: 588ms
643:    learn: 0.3029507    total: 1.06s    remaining: 586ms
644:    learn: 0.3027877    total: 1.06s    remaining: 585ms
645:    learn: 0.3027289    total: 1.06s    remaining: 583ms
646:    learn: 0.3026159    total: 1.06s    remaining: 581ms
647:    learn: 0.3025691    total: 1.07s    remaining: 580ms
648:    learn: 0.3024699    total: 1.07s    remaining: 578ms
649:    learn: 0.3023822    total: 1.07s    remaining: 576ms
650:    learn: 0.3022204    total: 1.07s    remaining: 574ms
651:    learn: 0.3021950    total: 1.07s    remaining: 573ms
652:    learn: 0.3020537    total: 1.07s    remaining: 571ms
653:    learn: 0.3018853    total: 1.08s    remaining: 569ms
654:    learn: 0.3018510    total: 1.08s    remaining: 568ms
655:    learn: 0.3017033    total: 1.08s    remaining: 566ms
656:    learn: 0.3016264    total: 1.08s    remaining: 564ms
657:    learn: 0.3015334    total: 1.08s    remaining: 563ms
658:    learn: 0.3014958    total: 1.08s    remaining: 561ms
659:    learn: 0.3014106    total: 1.08s    remaining: 559ms
660:    learn: 0.3013390    total: 1.09s    remaining: 557ms
661:    learn: 0.3012686    total: 1.09s    remaining: 556ms
662:    learn: 0.3011331    total: 1.09s    remaining: 554ms
663:    learn: 0.3010920    total: 1.09s    remaining: 552ms
664:    learn: 0.3009207    total: 1.09s    remaining: 552ms
665:    learn: 0.3008078    total: 1.1s    remaining: 550ms
666:    learn: 0.3007882    total: 1.1s    remaining: 549ms
667:    learn: 0.3007260    total: 1.1s    remaining: 548ms
668:    learn: 0.3006588    total: 1.1s    remaining: 546ms
669:    learn: 0.3004912    total: 1.11s    remaining: 545ms
670:    learn: 0.3004723    total: 1.11s    remaining: 543ms
671:    learn: 0.3003003    total: 1.11s    remaining: 542ms
672:    learn: 0.3001678    total: 1.11s    remaining: 540ms
673:    learn: 0.3000326    total: 1.11s    remaining: 538ms
674:    learn: 0.2999521    total: 1.11s    remaining: 537ms
675:    learn: 0.2997049    total: 1.12s    remaining: 535ms
676:    learn: 0.2995324    total: 1.12s    remaining: 533ms
677:    learn: 0.2994917    total: 1.12s    remaining: 531ms
678:    learn: 0.2994258    total: 1.12s    remaining: 530ms
679:    learn: 0.2992969    total: 1.12s    remaining: 528ms
680:    learn: 0.2991673    total: 1.12s    remaining: 527ms
681:    learn: 0.2990420    total: 1.13s    remaining: 525ms
682:    learn: 0.2989613    total: 1.13s    remaining: 523ms
683:    learn: 0.2988837    total: 1.13s    remaining: 521ms
684:    learn: 0.2988569    total: 1.13s    remaining: 520ms
685:    learn: 0.2988010    total: 1.13s    remaining: 518ms
686:    learn: 0.2986281    total: 1.13s    remaining: 516ms
687:    learn: 0.2985230    total: 1.14s    remaining: 515ms
688:    learn: 0.2984262    total: 1.14s    remaining: 513ms
689:    learn: 0.2983868    total: 1.14s    remaining: 511ms
690:    learn: 0.2983422    total: 1.14s    remaining: 510ms
691:    learn: 0.2982868    total: 1.14s    remaining: 508ms
692:    learn: 0.2981226    total: 1.14s    remaining: 507ms
693:    learn: 0.2980691    total: 1.15s    remaining: 505ms
694:    learn: 0.2980002    total: 1.15s    remaining: 504ms
695:    learn: 0.2977908    total: 1.15s    remaining: 502ms
696:    learn: 0.2977052    total: 1.15s    remaining: 500ms
697:    learn: 0.2976894    total: 1.15s    remaining: 498ms
698:    learn: 0.2975668    total: 1.15s    remaining: 497ms
699:    learn: 0.2974088    total: 1.16s    remaining: 495ms
700:    learn: 0.2972473    total: 1.16s    remaining: 493ms
701:    learn: 0.2971332    total: 1.16s    remaining: 492ms
702:    learn: 0.2970478    total: 1.16s    remaining: 490ms
703:    learn: 0.2969479    total: 1.16s    remaining: 488ms
704:    learn: 0.2968603    total: 1.16s    remaining: 486ms
705:    learn: 0.2968035    total: 1.16s    remaining: 485ms
706:    learn: 0.2967609    total: 1.17s    remaining: 483ms
707:    learn: 0.2965663    total: 1.17s    remaining: 481ms
708:    learn: 0.2963934    total: 1.17s    remaining: 479ms
709:    learn: 0.2961962    total: 1.17s    remaining: 478ms
710:    learn: 0.2961123    total: 1.17s    remaining: 476ms
711:    learn: 0.2960399    total: 1.17s    remaining: 474ms
712:    learn: 0.2959317    total: 1.17s    remaining: 473ms
713:    learn: 0.2957971    total: 1.18s    remaining: 471ms
714:    learn: 0.2956215    total: 1.18s    remaining: 469ms
715:    learn: 0.2954575    total: 1.18s    remaining: 468ms
716:    learn: 0.2953437    total: 1.18s    remaining: 466ms
717:    learn: 0.2952515    total: 1.18s    remaining: 465ms
718:    learn: 0.2951054    total: 1.18s    remaining: 463ms
719:    learn: 0.2950176    total: 1.19s    remaining: 461ms
720:    learn: 0.2949562    total: 1.19s    remaining: 460ms
721:    learn: 0.2948627    total: 1.19s    remaining: 458ms
722:    learn: 0.2946604    total: 1.19s    remaining: 456ms
723:    learn: 0.2945315    total: 1.19s    remaining: 455ms
724:    learn: 0.2943817    total: 1.2s    remaining: 453ms
725:    learn: 0.2942896    total: 1.2s    remaining: 452ms
726:    learn: 0.2942133    total: 1.2s    remaining: 452ms
727:    learn: 0.2941519    total: 1.21s    remaining: 451ms
728:    learn: 0.2940815    total: 1.21s    remaining: 450ms
729:    learn: 0.2939787    total: 1.21s    remaining: 449ms
730:    learn: 0.2938856    total: 1.22s    remaining: 447ms
731:    learn: 0.2937554    total: 1.22s    remaining: 446ms
732:    learn: 0.2936156    total: 1.22s    remaining: 444ms
733:    learn: 0.2934647    total: 1.22s    remaining: 442ms
734:    learn: 0.2933211    total: 1.22s    remaining: 441ms
735:    learn: 0.2932438    total: 1.23s    remaining: 440ms
736:    learn: 0.2931398    total: 1.23s    remaining: 439ms
737:    learn: 0.2929100    total: 1.23s    remaining: 437ms
738:    learn: 0.2927724    total: 1.23s    remaining: 436ms
739:    learn: 0.2926515    total: 1.24s    remaining: 434ms
740:    learn: 0.2924777    total: 1.24s    remaining: 432ms
741:    learn: 0.2922992    total: 1.24s    remaining: 430ms
742:    learn: 0.2922210    total: 1.24s    remaining: 429ms
743:    learn: 0.2919906    total: 1.24s    remaining: 427ms
744:    learn: 0.2919204    total: 1.24s    remaining: 425ms
745:    learn: 0.2917676    total: 1.24s    remaining: 424ms
746:    learn: 0.2917028    total: 1.25s    remaining: 422ms
747:    learn: 0.2916301    total: 1.25s    remaining: 420ms
748:    learn: 0.2915584    total: 1.25s    remaining: 418ms
749:    learn: 0.2914372    total: 1.25s    remaining: 416ms
750:    learn: 0.2913890    total: 1.25s    remaining: 415ms
751:    learn: 0.2912267    total: 1.25s    remaining: 413ms
752:    learn: 0.2910140    total: 1.25s    remaining: 411ms
753:    learn: 0.2908936    total: 1.25s    remaining: 409ms
754:    learn: 0.2907613    total: 1.25s    remaining: 407ms
755:    learn: 0.2906129    total: 1.26s    remaining: 406ms
756:    learn: 0.2905391    total: 1.26s    remaining: 404ms
757:    learn: 0.2903551    total: 1.26s    remaining: 402ms
758:    learn: 0.2902709    total: 1.26s    remaining: 400ms
759:    learn: 0.2901705    total: 1.26s    remaining: 398ms
760:    learn: 0.2899778    total: 1.26s    remaining: 397ms
761:    learn: 0.2898210    total: 1.26s    remaining: 395ms
762:    learn: 0.2897040    total: 1.26s    remaining: 393ms
763:    learn: 0.2896362    total: 1.27s    remaining: 391ms
764:    learn: 0.2895478    total: 1.27s    remaining: 390ms
765:    learn: 0.2894245    total: 1.27s    remaining: 388ms
766:    learn: 0.2893510    total: 1.27s    remaining: 386ms
767:    learn: 0.2891791    total: 1.27s    remaining: 384ms
768:    learn: 0.2891680    total: 1.27s    remaining: 383ms
769:    learn: 0.2890059    total: 1.27s    remaining: 381ms
770:    learn: 0.2889026    total: 1.28s    remaining: 379ms
771:    learn: 0.2886843    total: 1.28s    remaining: 377ms
772:    learn: 0.2885872    total: 1.28s    remaining: 376ms
773:    learn: 0.2884660    total: 1.28s    remaining: 374ms
774:    learn: 0.2883364    total: 1.28s    remaining: 372ms
775:    learn: 0.2882417    total: 1.28s    remaining: 371ms
776:    learn: 0.2881458    total: 1.29s    remaining: 369ms
777:    learn: 0.2880760    total: 1.29s    remaining: 367ms
778:    learn: 0.2880472    total: 1.29s    remaining: 366ms
779:    learn: 0.2879450    total: 1.29s    remaining: 364ms
780:    learn: 0.2877392    total: 1.29s    remaining: 362ms
781:    learn: 0.2876009    total: 1.29s    remaining: 361ms
782:    learn: 0.2873887    total: 1.29s    remaining: 359ms
783:    learn: 0.2872053    total: 1.3s    remaining: 358ms
784:    learn: 0.2870761    total: 1.3s    remaining: 356ms
785:    learn: 0.2869364    total: 1.3s    remaining: 355ms
786:    learn: 0.2868042    total: 1.31s    remaining: 354ms
787:    learn: 0.2866404    total: 1.31s    remaining: 352ms
788:    learn: 0.2864998    total: 1.31s    remaining: 351ms
789:    learn: 0.2863503    total: 1.31s    remaining: 349ms
790:    learn: 0.2862753    total: 1.31s    remaining: 347ms
791:    learn: 0.2861606    total: 1.31s    remaining: 346ms
792:    learn: 0.2861053    total: 1.32s    remaining: 344ms
793:    learn: 0.2859749    total: 1.32s    remaining: 342ms
794:    learn: 0.2858847    total: 1.32s    remaining: 340ms
795:    learn: 0.2857806    total: 1.32s    remaining: 339ms
796:    learn: 0.2857007    total: 1.32s    remaining: 337ms
797:    learn: 0.2856353    total: 1.32s    remaining: 335ms
798:    learn: 0.2855295    total: 1.33s    remaining: 334ms
799:    learn: 0.2854726    total: 1.33s    remaining: 332ms
800:    learn: 0.2854507    total: 1.33s    remaining: 330ms
801:    learn: 0.2853060    total: 1.33s    remaining: 329ms
802:    learn: 0.2852216    total: 1.33s    remaining: 327ms
803:    learn: 0.2851484    total: 1.33s    remaining: 325ms
804:    learn: 0.2850786    total: 1.33s    remaining: 324ms
805:    learn: 0.2850163    total: 1.34s    remaining: 322ms
806:    learn: 0.2849433    total: 1.34s    remaining: 320ms
807:    learn: 0.2848859    total: 1.34s    remaining: 319ms
808:    learn: 0.2847031    total: 1.34s    remaining: 317ms
809:    learn: 0.2846279    total: 1.34s    remaining: 315ms
810:    learn: 0.2844524    total: 1.35s    remaining: 314ms
811:    learn: 0.2843816    total: 1.35s    remaining: 312ms
812:    learn: 0.2842996    total: 1.35s    remaining: 310ms
813:    learn: 0.2841085    total: 1.35s    remaining: 309ms
814:    learn: 0.2839898    total: 1.35s    remaining: 307ms
815:    learn: 0.2839256    total: 1.35s    remaining: 305ms
816:    learn: 0.2838326    total: 1.36s    remaining: 304ms
817:    learn: 0.2837922    total: 1.36s    remaining: 302ms
818:    learn: 0.2837661    total: 1.36s    remaining: 300ms
819:    learn: 0.2837009    total: 1.36s    remaining: 299ms
820:    learn: 0.2836201    total: 1.36s    remaining: 297ms
821:    learn: 0.2834531    total: 1.36s    remaining: 295ms
822:    learn: 0.2833575    total: 1.36s    remaining: 294ms
823:    learn: 0.2832791    total: 1.37s    remaining: 292ms
824:    learn: 0.2832314    total: 1.37s    remaining: 290ms
825:    learn: 0.2831767    total: 1.37s    remaining: 289ms
826:    learn: 0.2830808    total: 1.37s    remaining: 287ms
827:    learn: 0.2829269    total: 1.37s    remaining: 286ms
828:    learn: 0.2828895    total: 1.38s    remaining: 284ms
829:    learn: 0.2828296    total: 1.38s    remaining: 282ms
830:    learn: 0.2826825    total: 1.38s    remaining: 281ms
831:    learn: 0.2825853    total: 1.38s    remaining: 279ms
832:    learn: 0.2824905    total: 1.39s    remaining: 278ms
833:    learn: 0.2823213    total: 1.39s    remaining: 276ms
834:    learn: 0.2822432    total: 1.39s    remaining: 274ms
835:    learn: 0.2821129    total: 1.39s    remaining: 273ms
836:    learn: 0.2820086    total: 1.39s    remaining: 271ms
837:    learn: 0.2819346    total: 1.39s    remaining: 269ms
838:    learn: 0.2817989    total: 1.4s    remaining: 268ms
839:    learn: 0.2817084    total: 1.4s    remaining: 266ms
840:    learn: 0.2813877    total: 1.4s    remaining: 265ms
841:    learn: 0.2812499    total: 1.4s    remaining: 263ms
842:    learn: 0.2811951    total: 1.41s    remaining: 262ms
843:    learn: 0.2810829    total: 1.41s    remaining: 260ms
844:    learn: 0.2810392    total: 1.41s    remaining: 259ms
845:    learn: 0.2809627    total: 1.41s    remaining: 257ms
846:    learn: 0.2808334    total: 1.41s    remaining: 255ms
847:    learn: 0.2807824    total: 1.41s    remaining: 253ms
848:    learn: 0.2807220    total: 1.41s    remaining: 252ms
849:    learn: 0.2806410    total: 1.42s    remaining: 251ms
850:    learn: 0.2805382    total: 1.43s    remaining: 250ms
851:    learn: 0.2804362    total: 1.43s    remaining: 249ms
852:    learn: 0.2803172    total: 1.44s    remaining: 248ms
853:    learn: 0.2802016    total: 1.44s    remaining: 247ms
854:    learn: 0.2800346    total: 1.45s    remaining: 245ms
855:    learn: 0.2800234    total: 1.45s    remaining: 244ms
856:    learn: 0.2798293    total: 1.45s    remaining: 242ms
857:    learn: 0.2797126    total: 1.45s    remaining: 240ms
858:    learn: 0.2795744    total: 1.45s    remaining: 238ms
859:    learn: 0.2794426    total: 1.45s    remaining: 237ms
860:    learn: 0.2791854    total: 1.46s    remaining: 235ms
861:    learn: 0.2791077    total: 1.46s    remaining: 233ms
862:    learn: 0.2790048    total: 1.46s    remaining: 232ms
863:    learn: 0.2788733    total: 1.46s    remaining: 230ms
864:    learn: 0.2788182    total: 1.46s    remaining: 228ms
865:    learn: 0.2786590    total: 1.46s    remaining: 227ms
866:    learn: 0.2785414    total: 1.47s    remaining: 225ms
867:    learn: 0.2784293    total: 1.47s    remaining: 223ms
868:    learn: 0.2783252    total: 1.47s    remaining: 221ms
869:    learn: 0.2781887    total: 1.47s    remaining: 220ms
870:    learn: 0.2779748    total: 1.47s    remaining: 218ms
871:    learn: 0.2779000    total: 1.47s    remaining: 216ms
872:    learn: 0.2778225    total: 1.48s    remaining: 215ms
873:    learn: 0.2777576    total: 1.48s    remaining: 213ms
874:    learn: 0.2777006    total: 1.48s    remaining: 211ms
875:    learn: 0.2776684    total: 1.48s    remaining: 210ms
876:    learn: 0.2776048    total: 1.49s    remaining: 208ms
877:    learn: 0.2775366    total: 1.49s    remaining: 207ms
878:    learn: 0.2774703    total: 1.49s    remaining: 205ms
879:    learn: 0.2773172    total: 1.49s    remaining: 203ms
880:    learn: 0.2772617    total: 1.49s    remaining: 202ms
881:    learn: 0.2771427    total: 1.49s    remaining: 200ms
882:    learn: 0.2770267    total: 1.5s    remaining: 198ms
883:    learn: 0.2769163    total: 1.5s    remaining: 196ms
884:    learn: 0.2767768    total: 1.5s    remaining: 195ms
885:    learn: 0.2767300    total: 1.5s    remaining: 193ms
886:    learn: 0.2766843    total: 1.5s    remaining: 191ms
887:    learn: 0.2765526    total: 1.5s    remaining: 190ms
888:    learn: 0.2764557    total: 1.5s    remaining: 188ms
889:    learn: 0.2763369    total: 1.51s    remaining: 186ms
890:    learn: 0.2762687    total: 1.51s    remaining: 184ms
891:    learn: 0.2762030    total: 1.51s    remaining: 183ms
892:    learn: 0.2761240    total: 1.51s    remaining: 181ms
893:    learn: 0.2760416    total: 1.51s    remaining: 179ms
894:    learn: 0.2759239    total: 1.51s    remaining: 178ms
895:    learn: 0.2757411    total: 1.52s    remaining: 176ms
896:    learn: 0.2756781    total: 1.52s    remaining: 174ms
897:    learn: 0.2755656    total: 1.52s    remaining: 173ms
898:    learn: 0.2755017    total: 1.53s    remaining: 172ms
899:    learn: 0.2753771    total: 1.53s    remaining: 170ms
900:    learn: 0.2752091    total: 1.53s    remaining: 168ms
901:    learn: 0.2751239    total: 1.53s    remaining: 167ms
902:    learn: 0.2750431    total: 1.53s    remaining: 165ms
903:    learn: 0.2748063    total: 1.53s    remaining: 163ms
904:    learn: 0.2747393    total: 1.54s    remaining: 161ms
905:    learn: 0.2747020    total: 1.54s    remaining: 160ms
906:    learn: 0.2746334    total: 1.54s    remaining: 158ms
907:    learn: 0.2744415    total: 1.54s    remaining: 156ms
908:    learn: 0.2743686    total: 1.54s    remaining: 155ms
909:    learn: 0.2742677    total: 1.55s    remaining: 153ms
910:    learn: 0.2741824    total: 1.55s    remaining: 151ms
911:    learn: 0.2741178    total: 1.55s    remaining: 150ms
912:    learn: 0.2739814    total: 1.55s    remaining: 148ms
913:    learn: 0.2737799    total: 1.55s    remaining: 146ms
914:    learn: 0.2736915    total: 1.55s    remaining: 144ms
915:    learn: 0.2736188    total: 1.56s    remaining: 143ms
916:    learn: 0.2734803    total: 1.56s    remaining: 142ms
917:    learn: 0.2733654    total: 1.57s    remaining: 140ms
918:    learn: 0.2732999    total: 1.57s    remaining: 138ms
919:    learn: 0.2731856    total: 1.57s    remaining: 136ms
920:    learn: 0.2731466    total: 1.57s    remaining: 135ms
921:    learn: 0.2730750    total: 1.57s    remaining: 133ms
922:    learn: 0.2729086    total: 1.57s    remaining: 131ms
923:    learn: 0.2728371    total: 1.57s    remaining: 130ms
924:    learn: 0.2727040    total: 1.58s    remaining: 128ms
925:    learn: 0.2726765    total: 1.58s    remaining: 126ms
926:    learn: 0.2726018    total: 1.58s    remaining: 124ms
927:    learn: 0.2725268    total: 1.58s    remaining: 123ms
928:    learn: 0.2723675    total: 1.58s    remaining: 121ms
929:    learn: 0.2723010    total: 1.58s    remaining: 119ms
930:    learn: 0.2721701    total: 1.59s    remaining: 118ms
931:    learn: 0.2721272    total: 1.59s    remaining: 116ms
932:    learn: 0.2720654    total: 1.59s    remaining: 115ms
933:    learn: 0.2717534    total: 1.6s    remaining: 113ms
934:    learn: 0.2716384    total: 1.6s    remaining: 111ms
935:    learn: 0.2714989    total: 1.6s    remaining: 109ms
936:    learn: 0.2712303    total: 1.6s    remaining: 108ms
937:    learn: 0.2711671    total: 1.6s    remaining: 106ms
938:    learn: 0.2711033    total: 1.6s    remaining: 104ms
939:    learn: 0.2709477    total: 1.61s    remaining: 103ms
940:    learn: 0.2708769    total: 1.61s    remaining: 101ms
941:    learn: 0.2705752    total: 1.61s    remaining: 99.2ms
942:    learn: 0.2704340    total: 1.61s    remaining: 97.5ms
943:    learn: 0.2702520    total: 1.61s    remaining: 95.8ms
944:    learn: 0.2701982    total: 1.62s    remaining: 94.1ms
945:    learn: 0.2700223    total: 1.62s    remaining: 92.3ms
946:    learn: 0.2699653    total: 1.62s    remaining: 90.6ms
947:    learn: 0.2698782    total: 1.62s    remaining: 88.9ms
948:    learn: 0.2698224    total: 1.62s    remaining: 87.2ms
949:    learn: 0.2697586    total: 1.62s    remaining: 85.5ms
950:    learn: 0.2696519    total: 1.63s    remaining: 83.8ms
951:    learn: 0.2695586    total: 1.63s    remaining: 82.1ms
952:    learn: 0.2694578    total: 1.63s    remaining: 80.3ms
953:    learn: 0.2693905    total: 1.63s    remaining: 78.6ms
954:    learn: 0.2693518    total: 1.63s    remaining: 76.9ms
955:    learn: 0.2692253    total: 1.63s    remaining: 75.2ms
956:    learn: 0.2690230    total: 1.64s    remaining: 73.5ms
957:    learn: 0.2689603    total: 1.64s    remaining: 71.8ms
958:    learn: 0.2689104    total: 1.64s    remaining: 70ms
959:    learn: 0.2688855    total: 1.64s    remaining: 68.3ms
960:    learn: 0.2688177    total: 1.64s    remaining: 66.6ms
961:    learn: 0.2686715    total: 1.64s    remaining: 64.9ms
962:    learn: 0.2686072    total: 1.64s    remaining: 63.2ms
963:    learn: 0.2685623    total: 1.65s    remaining: 61.5ms
964:    learn: 0.2685164    total: 1.65s    remaining: 59.7ms
965:    learn: 0.2683830    total: 1.65s    remaining: 58ms
966:    learn: 0.2683233    total: 1.65s    remaining: 56.3ms
967:    learn: 0.2681680    total: 1.65s    remaining: 54.6ms
968:    learn: 0.2680945    total: 1.65s    remaining: 52.9ms
969:    learn: 0.2680299    total: 1.66s    remaining: 51.2ms
970:    learn: 0.2679874    total: 1.66s    remaining: 49.5ms
971:    learn: 0.2678332    total: 1.66s    remaining: 47.8ms
972:    learn: 0.2677245    total: 1.66s    remaining: 46.1ms
973:    learn: 0.2675624    total: 1.66s    remaining: 44.4ms
974:    learn: 0.2674923    total: 1.66s    remaining: 42.6ms
975:    learn: 0.2674601    total: 1.66s    remaining: 40.9ms
976:    learn: 0.2674036    total: 1.67s    remaining: 39.2ms
977:    learn: 0.2673396    total: 1.67s    remaining: 37.5ms
978:    learn: 0.2672855    total: 1.67s    remaining: 35.8ms
979:    learn: 0.2672348    total: 1.67s    remaining: 34.1ms
980:    learn: 0.2671636    total: 1.67s    remaining: 32.4ms
981:    learn: 0.2671071    total: 1.67s    remaining: 30.7ms
982:    learn: 0.2670526    total: 1.68s    remaining: 29ms
983:    learn: 0.2670075    total: 1.68s    remaining: 27.3ms
984:    learn: 0.2669261    total: 1.68s    remaining: 25.6ms
985:    learn: 0.2668792    total: 1.68s    remaining: 23.9ms
986:    learn: 0.2668012    total: 1.68s    remaining: 22.2ms
987:    learn: 0.2667278    total: 1.68s    remaining: 20.4ms
988:    learn: 0.2665946    total: 1.69s    remaining: 18.7ms
989:    learn: 0.2664965    total: 1.69s    remaining: 17ms
990:    learn: 0.2663784    total: 1.69s    remaining: 15.3ms
991:    learn: 0.2663341    total: 1.69s    remaining: 13.6ms
992:    learn: 0.2662716    total: 1.69s    remaining: 11.9ms
993:    learn: 0.2661899    total: 1.69s    remaining: 10.2ms
994:    learn: 0.2660623    total: 1.69s    remaining: 8.51ms
995:    learn: 0.2660156    total: 1.7s    remaining: 6.81ms
996:    learn: 0.2659742    total: 1.7s    remaining: 5.11ms
997:    learn: 0.2658469    total: 1.7s    remaining: 3.4ms
998:    learn: 0.2657372    total: 1.7s    remaining: 1.7ms
999:    learn: 0.2656287    total: 1.7s    remaining: 0us

정확도:  0.9216417910447762
CPU times: user 2.54 s, sys: 282 ms, total: 2.82 s
Wall time: 2.04 s

하이퍼파라미터 튜닝 후

%%time
model_cb2=cb.CatBoostClassifier(depth= 13,learning_rate= 0.1)
model_cb2.fit(X_train, y_train)
y_pred_cb2=model_cb2.predict(X1_test)
print("\n정확도: ", metrics.accuracy_score(y1_test, y_pred_cb2))

0:    learn: 0.6291440    total: 1.36ms    remaining: 1.36s
1:    learn: 0.5748476    total: 6.07ms    remaining: 3.03s
2:    learn: 0.5279115    total: 65.1ms    remaining: 21.6s
3:    learn: 0.4926697    total: 67.6ms    remaining: 16.8s
4:    learn: 0.4642216    total: 83.3ms    remaining: 16.6s
5:    learn: 0.4382921    total: 144ms    remaining: 23.9s
6:    learn: 0.4215442    total: 159ms    remaining: 22.6s
7:    learn: 0.4041541    total: 179ms    remaining: 22.2s
8:    learn: 0.3930502    total: 181ms    remaining: 20s
9:    learn: 0.3799636    total: 241ms    remaining: 23.9s
10:    learn: 0.3699306    total: 257ms    remaining: 23.1s
11:    learn: 0.3651040    total: 259ms    remaining: 21.3s
12:    learn: 0.3566676    total: 316ms    remaining: 24s
13:    learn: 0.3551631    total: 317ms    remaining: 22.3s
14:    learn: 0.3513713    total: 319ms    remaining: 21s
15:    learn: 0.3442301    total: 389ms    remaining: 23.9s
16:    learn: 0.3419305    total: 394ms    remaining: 22.8s
17:    learn: 0.3397292    total: 396ms    remaining: 21.6s
18:    learn: 0.3385750    total: 397ms    remaining: 20.5s
19:    learn: 0.3357366    total: 400ms    remaining: 19.6s
20:    learn: 0.3336362    total: 402ms    remaining: 18.7s
21:    learn: 0.3320698    total: 403ms    remaining: 17.9s
22:    learn: 0.3246891    total: 484ms    remaining: 20.6s
23:    learn: 0.3184729    total: 561ms    remaining: 22.8s
24:    learn: 0.3148940    total: 567ms    remaining: 22.1s
25:    learn: 0.3147399    total: 568ms    remaining: 21.3s
26:    learn: 0.3102915    total: 612ms    remaining: 22s
27:    learn: 0.3079201    total: 622ms    remaining: 21.6s
28:    learn: 0.3016318    total: 707ms    remaining: 23.7s
29:    learn: 0.2984929    total: 794ms    remaining: 25.7s
30:    learn: 0.2959946    total: 882ms    remaining: 27.6s
31:    learn: 0.2956280    total: 884ms    remaining: 26.7s
32:    learn: 0.2925718    total: 927ms    remaining: 27.2s
33:    learn: 0.2875422    total: 1.01s    remaining: 28.6s
34:    learn: 0.2860851    total: 1.01s    remaining: 27.9s
35:    learn: 0.2829340    total: 1.03s    remaining: 27.7s
36:    learn: 0.2789452    total: 1.12s    remaining: 29.1s
37:    learn: 0.2783872    total: 1.12s    remaining: 28.4s
38:    learn: 0.2749716    total: 1.21s    remaining: 29.7s
39:    learn: 0.2719646    total: 1.28s    remaining: 30.9s
40:    learn: 0.2709034    total: 1.29s    remaining: 30.2s
41:    learn: 0.2673618    total: 1.38s    remaining: 31.4s
42:    learn: 0.2642811    total: 1.46s    remaining: 32.4s
43:    learn: 0.2637187    total: 1.46s    remaining: 31.7s
44:    learn: 0.2635482    total: 1.46s    remaining: 31s
45:    learn: 0.2630957    total: 1.46s    remaining: 30.3s
46:    learn: 0.2606977    total: 1.54s    remaining: 31.2s
47:    learn: 0.2588854    total: 1.62s    remaining: 32.1s
48:    learn: 0.2582335    total: 1.62s    remaining: 31.4s
49:    learn: 0.2562634    total: 1.64s    remaining: 31.1s
50:    learn: 0.2533263    total: 1.72s    remaining: 32s
51:    learn: 0.2517890    total: 1.74s    remaining: 31.7s
52:    learn: 0.2509030    total: 1.74s    remaining: 31.2s
53:    learn: 0.2476277    total: 1.83s    remaining: 32.1s
54:    learn: 0.2450707    total: 1.91s    remaining: 32.9s
55:    learn: 0.2450066    total: 1.91s    remaining: 32.3s
56:    learn: 0.2445548    total: 1.93s    remaining: 31.9s
57:    learn: 0.2425602    total: 2s    remaining: 32.6s
58:    learn: 0.2424471    total: 2s    remaining: 32s
59:    learn: 0.2394531    total: 2.09s    remaining: 32.8s
60:    learn: 0.2382583    total: 2.12s    remaining: 32.6s
61:    learn: 0.2374918    total: 2.12s    remaining: 32s
62:    learn: 0.2336260    total: 2.2s    remaining: 32.8s
63:    learn: 0.2329741    total: 2.24s    remaining: 32.8s
64:    learn: 0.2325789    total: 2.24s    remaining: 32.3s
65:    learn: 0.2299386    total: 2.32s    remaining: 32.9s
66:    learn: 0.2281529    total: 2.42s    remaining: 33.6s
67:    learn: 0.2257874    total: 2.46s    remaining: 33.7s
68:    learn: 0.2232054    total: 2.54s    remaining: 34.3s
69:    learn: 0.2222198    total: 2.55s    remaining: 33.9s
70:    learn: 0.2199046    total: 2.63s    remaining: 34.4s
71:    learn: 0.2177259    total: 2.71s    remaining: 35s
72:    learn: 0.2144199    total: 2.8s    remaining: 35.6s
73:    learn: 0.2127443    total: 2.89s    remaining: 36.1s
74:    learn: 0.2111802    total: 2.96s    remaining: 36.6s
75:    learn: 0.2091821    total: 3.04s    remaining: 37s
76:    learn: 0.2066723    total: 3.13s    remaining: 37.5s
77:    learn: 0.2058984    total: 3.15s    remaining: 37.3s
78:    learn: 0.2048881    total: 3.17s    remaining: 37s
79:    learn: 0.2031754    total: 3.26s    remaining: 37.4s
80:    learn: 0.2030224    total: 3.26s    remaining: 37s
81:    learn: 0.2014270    total: 3.34s    remaining: 37.4s
82:    learn: 0.1992327    total: 3.44s    remaining: 38s
83:    learn: 0.1974043    total: 3.52s    remaining: 38.4s
84:    learn: 0.1969113    total: 3.55s    remaining: 38.2s
85:    learn: 0.1952535    total: 3.63s    remaining: 38.6s
86:    learn: 0.1949188    total: 3.63s    remaining: 38.1s
87:    learn: 0.1941748    total: 3.71s    remaining: 38.5s
88:    learn: 0.1933274    total: 3.79s    remaining: 38.8s
89:    learn: 0.1920005    total: 3.88s    remaining: 39.2s
90:    learn: 0.1902233    total: 3.96s    remaining: 39.5s
91:    learn: 0.1886434    total: 4.04s    remaining: 39.9s
92:    learn: 0.1878609    total: 4.12s    remaining: 40.2s
93:    learn: 0.1873971    total: 4.14s    remaining: 39.9s
94:    learn: 0.1860442    total: 4.22s    remaining: 40.2s
95:    learn: 0.1841309    total: 4.3s    remaining: 40.5s
96:    learn: 0.1836029    total: 4.31s    remaining: 40.1s
97:    learn: 0.1830616    total: 4.39s    remaining: 40.5s
98:    learn: 0.1818786    total: 4.47s    remaining: 40.7s
99:    learn: 0.1812028    total: 4.52s    remaining: 40.7s
100:    learn: 0.1801801    total: 4.59s    remaining: 40.9s
101:    learn: 0.1789388    total: 4.68s    remaining: 41.2s
102:    learn: 0.1772152    total: 4.76s    remaining: 41.4s
103:    learn: 0.1766453    total: 4.84s    remaining: 41.7s
104:    learn: 0.1749187    total: 4.92s    remaining: 41.9s
105:    learn: 0.1742036    total: 5s    remaining: 42.1s
106:    learn: 0.1726317    total: 5.08s    remaining: 42.4s
107:    learn: 0.1708060    total: 5.17s    remaining: 42.7s
108:    learn: 0.1694547    total: 5.25s    remaining: 42.9s
109:    learn: 0.1689197    total: 5.33s    remaining: 43.1s
110:    learn: 0.1674540    total: 5.41s    remaining: 43.3s
111:    learn: 0.1665666    total: 5.49s    remaining: 43.5s
112:    learn: 0.1655337    total: 5.57s    remaining: 43.7s
113:    learn: 0.1640463    total: 5.65s    remaining: 43.9s
114:    learn: 0.1632857    total: 5.73s    remaining: 44.1s
115:    learn: 0.1624247    total: 5.81s    remaining: 44.3s
116:    learn: 0.1609198    total: 5.89s    remaining: 44.5s
117:    learn: 0.1599200    total: 5.97s    remaining: 44.7s
118:    learn: 0.1588361    total: 6.05s    remaining: 44.8s
119:    learn: 0.1579644    total: 6.06s    remaining: 44.4s
120:    learn: 0.1569607    total: 6.14s    remaining: 44.6s
121:    learn: 0.1560274    total: 6.22s    remaining: 44.7s
122:    learn: 0.1551732    total: 6.29s    remaining: 44.9s
123:    learn: 0.1544019    total: 6.38s    remaining: 45s
124:    learn: 0.1537151    total: 6.46s    remaining: 45.3s
125:    learn: 0.1511900    total: 6.53s    remaining: 45.3s
126:    learn: 0.1507331    total: 6.59s    remaining: 45.3s
127:    learn: 0.1499530    total: 6.65s    remaining: 45.3s
128:    learn: 0.1490988    total: 6.72s    remaining: 45.4s
129:    learn: 0.1484887    total: 6.78s    remaining: 45.4s
130:    learn: 0.1476770    total: 6.84s    remaining: 45.4s
131:    learn: 0.1465705    total: 6.9s    remaining: 45.4s
132:    learn: 0.1462378    total: 6.93s    remaining: 45.2s
133:    learn: 0.1457998    total: 6.99s    remaining: 45.2s
134:    learn: 0.1452223    total: 7s    remaining: 44.8s
135:    learn: 0.1439411    total: 7.05s    remaining: 44.8s
136:    learn: 0.1435812    total: 7.11s    remaining: 44.8s
137:    learn: 0.1428338    total: 7.18s    remaining: 44.8s
138:    learn: 0.1416907    total: 7.24s    remaining: 44.8s
139:    learn: 0.1414533    total: 7.24s    remaining: 44.5s
140:    learn: 0.1405029    total: 7.3s    remaining: 44.5s
141:    learn: 0.1398655    total: 7.36s    remaining: 44.5s
142:    learn: 0.1388855    total: 7.43s    remaining: 44.5s
143:    learn: 0.1382867    total: 7.49s    remaining: 44.5s
144:    learn: 0.1376590    total: 7.5s    remaining: 44.3s
145:    learn: 0.1368349    total: 7.58s    remaining: 44.4s
146:    learn: 0.1356921    total: 7.67s    remaining: 44.5s
147:    learn: 0.1354803    total: 7.67s    remaining: 44.1s
148:    learn: 0.1346190    total: 7.75s    remaining: 44.3s
149:    learn: 0.1341611    total: 7.83s    remaining: 44.4s
150:    learn: 0.1334851    total: 7.92s    remaining: 44.5s
151:    learn: 0.1329815    total: 8s    remaining: 44.6s
152:    learn: 0.1325443    total: 8.03s    remaining: 44.5s
153:    learn: 0.1314231    total: 8.12s    remaining: 44.6s
154:    learn: 0.1303450    total: 8.14s    remaining: 44.4s
155:    learn: 0.1300380    total: 8.22s    remaining: 44.5s
156:    learn: 0.1291482    total: 8.29s    remaining: 44.5s
157:    learn: 0.1285500    total: 8.38s    remaining: 44.6s
158:    learn: 0.1276863    total: 8.47s    remaining: 44.8s
159:    learn: 0.1267840    total: 8.54s    remaining: 44.9s
160:    learn: 0.1261672    total: 8.63s    remaining: 45s
161:    learn: 0.1258289    total: 8.65s    remaining: 44.7s
162:    learn: 0.1256662    total: 8.69s    remaining: 44.6s
163:    learn: 0.1251198    total: 8.77s    remaining: 44.7s
164:    learn: 0.1243440    total: 8.85s    remaining: 44.8s
165:    learn: 0.1238743    total: 8.93s    remaining: 44.9s
166:    learn: 0.1232649    total: 9.01s    remaining: 45s
167:    learn: 0.1223070    total: 9.09s    remaining: 45s
168:    learn: 0.1222166    total: 9.17s    remaining: 45.1s
169:    learn: 0.1214763    total: 9.25s    remaining: 45.2s
170:    learn: 0.1208712    total: 9.33s    remaining: 45.2s
171:    learn: 0.1203816    total: 9.35s    remaining: 45s
172:    learn: 0.1195046    total: 9.43s    remaining: 45.1s
173:    learn: 0.1191177    total: 9.52s    remaining: 45.2s
174:    learn: 0.1190539    total: 9.52s    remaining: 44.9s
175:    learn: 0.1186965    total: 9.6s    remaining: 45s
176:    learn: 0.1186388    total: 9.61s    remaining: 44.7s
177:    learn: 0.1180183    total: 9.69s    remaining: 44.7s
178:    learn: 0.1174797    total: 9.77s    remaining: 44.8s
179:    learn: 0.1170483    total: 9.87s    remaining: 45s
180:    learn: 0.1163883    total: 9.95s    remaining: 45s
181:    learn: 0.1161562    total: 9.99s    remaining: 44.9s
182:    learn: 0.1157569    total: 10s    remaining: 44.7s
183:    learn: 0.1157034    total: 10s    remaining: 44.4s
184:    learn: 0.1150241    total: 10.1s    remaining: 44.5s
185:    learn: 0.1143419    total: 10.2s    remaining: 44.5s
186:    learn: 0.1142538    total: 10.2s    remaining: 44.3s
187:    learn: 0.1138790    total: 10.3s    remaining: 44.3s
188:    learn: 0.1133401    total: 10.3s    remaining: 44.4s
189:    learn: 0.1132788    total: 10.3s    remaining: 44.1s
190:    learn: 0.1131678    total: 10.4s    remaining: 43.8s
191:    learn: 0.1125761    total: 10.4s    remaining: 43.9s
192:    learn: 0.1119667    total: 10.5s    remaining: 43.8s
193:    learn: 0.1116648    total: 10.6s    remaining: 43.9s
194:    learn: 0.1109593    total: 10.6s    remaining: 43.9s
195:    learn: 0.1102323    total: 10.7s    remaining: 44s
196:    learn: 0.1096026    total: 10.8s    remaining: 44s
197:    learn: 0.1095286    total: 10.8s    remaining: 43.8s
198:    learn: 0.1090764    total: 10.9s    remaining: 43.8s
199:    learn: 0.1085613    total: 11s    remaining: 43.9s
200:    learn: 0.1081235    total: 11s    remaining: 43.9s
201:    learn: 0.1075332    total: 11.1s    remaining: 44s
202:    learn: 0.1070773    total: 11.2s    remaining: 44s
203:    learn: 0.1066367    total: 11.3s    remaining: 44.1s
204:    learn: 0.1061256    total: 11.4s    remaining: 44.1s
205:    learn: 0.1057380    total: 11.5s    remaining: 44.1s
206:    learn: 0.1052409    total: 11.5s    remaining: 44.2s
207:    learn: 0.1049293    total: 11.6s    remaining: 44.2s
208:    learn: 0.1048501    total: 11.6s    remaining: 44s
209:    learn: 0.1046446    total: 11.7s    remaining: 43.9s
210:    learn: 0.1037971    total: 11.7s    remaining: 43.9s
211:    learn: 0.1035343    total: 11.8s    remaining: 44s
212:    learn: 0.1030647    total: 11.9s    remaining: 44s
213:    learn: 0.1027919    total: 12s    remaining: 44.1s
214:    learn: 0.1025205    total: 12.1s    remaining: 44.1s
215:    learn: 0.1022581    total: 12.2s    remaining: 44.2s
216:    learn: 0.1019889    total: 12.2s    remaining: 44.2s
217:    learn: 0.1017563    total: 12.3s    remaining: 44.2s
218:    learn: 0.1012053    total: 12.4s    remaining: 44.2s
219:    learn: 0.1008358    total: 12.5s    remaining: 44.3s
220:    learn: 0.1002939    total: 12.6s    remaining: 44.3s
221:    learn: 0.0998744    total: 12.7s    remaining: 44.3s
222:    learn: 0.0995125    total: 12.7s    remaining: 44.4s
223:    learn: 0.0994734    total: 12.8s    remaining: 44.2s
224:    learn: 0.0991801    total: 12.8s    remaining: 44.2s
225:    learn: 0.0989244    total: 12.9s    remaining: 44.2s
226:    learn: 0.0985547    total: 13s    remaining: 44.3s
227:    learn: 0.0981702    total: 13.1s    remaining: 44.3s
228:    learn: 0.0978596    total: 13.2s    remaining: 44.3s
229:    learn: 0.0975044    total: 13.2s    remaining: 44.4s
230:    learn: 0.0972430    total: 13.3s    remaining: 44.4s
231:    learn: 0.0972029    total: 13.3s    remaining: 44.2s
232:    learn: 0.0970625    total: 13.4s    remaining: 44.2s
233:    learn: 0.0969125    total: 13.5s    remaining: 44.2s
234:    learn: 0.0965909    total: 13.6s    remaining: 44.2s
235:    learn: 0.0961394    total: 13.7s    remaining: 44.3s
236:    learn: 0.0956679    total: 13.8s    remaining: 44.3s
237:    learn: 0.0954217    total: 13.8s    remaining: 44.3s
238:    learn: 0.0952190    total: 13.9s    remaining: 44.3s
239:    learn: 0.0946616    total: 14s    remaining: 44.3s
240:    learn: 0.0946270    total: 14s    remaining: 44.1s
241:    learn: 0.0944199    total: 14.1s    remaining: 44.1s
242:    learn: 0.0941707    total: 14.2s    remaining: 44.2s
243:    learn: 0.0939590    total: 14.3s    remaining: 44.2s
244:    learn: 0.0935693    total: 14.3s    remaining: 44.2s
245:    learn: 0.0932719    total: 14.4s    remaining: 44.2s
246:    learn: 0.0929236    total: 14.4s    remaining: 44s
247:    learn: 0.0925825    total: 14.5s    remaining: 44s
248:    learn: 0.0923408    total: 14.6s    remaining: 44s
249:    learn: 0.0921541    total: 14.7s    remaining: 44s
250:    learn: 0.0917607    total: 14.8s    remaining: 44s
251:    learn: 0.0916959    total: 14.8s    remaining: 43.8s
252:    learn: 0.0916654    total: 14.8s    remaining: 43.6s
253:    learn: 0.0914250    total: 14.9s    remaining: 43.6s
254:    learn: 0.0912275    total: 14.9s    remaining: 43.6s
255:    learn: 0.0909331    total: 15s    remaining: 43.5s
256:    learn: 0.0907212    total: 15s    remaining: 43.5s
257:    learn: 0.0903835    total: 15.1s    remaining: 43.5s
258:    learn: 0.0899619    total: 15.2s    remaining: 43.5s
259:    learn: 0.0895930    total: 15.3s    remaining: 43.5s
260:    learn: 0.0894562    total: 15.3s    remaining: 43.3s
261:    learn: 0.0891225    total: 15.4s    remaining: 43.3s
262:    learn: 0.0889105    total: 15.4s    remaining: 43.1s
263:    learn: 0.0887177    total: 15.5s    remaining: 43.1s
264:    learn: 0.0885528    total: 15.5s    remaining: 43s
265:    learn: 0.0884071    total: 15.5s    remaining: 42.8s
266:    learn: 0.0880970    total: 15.6s    remaining: 42.7s
267:    learn: 0.0878855    total: 15.6s    remaining: 42.7s
268:    learn: 0.0876066    total: 15.7s    remaining: 42.7s
269:    learn: 0.0874569    total: 15.8s    remaining: 42.6s
270:    learn: 0.0872114    total: 15.8s    remaining: 42.6s
271:    learn: 0.0868274    total: 15.9s    remaining: 42.5s
272:    learn: 0.0865278    total: 15.9s    remaining: 42.5s
273:    learn: 0.0862970    total: 16s    remaining: 42.4s
274:    learn: 0.0859278    total: 16.1s    remaining: 42.3s
275:    learn: 0.0857807    total: 16.1s    remaining: 42.3s
276:    learn: 0.0855715    total: 16.2s    remaining: 42.3s
277:    learn: 0.0853894    total: 16.2s    remaining: 42.1s
278:    learn: 0.0853644    total: 16.2s    remaining: 41.9s
279:    learn: 0.0851855    total: 16.3s    remaining: 41.8s
280:    learn: 0.0850113    total: 16.3s    remaining: 41.8s
281:    learn: 0.0848486    total: 16.4s    remaining: 41.7s
282:    learn: 0.0844737    total: 16.5s    remaining: 41.7s
283:    learn: 0.0841315    total: 16.5s    remaining: 41.6s
284:    learn: 0.0839427    total: 16.6s    remaining: 41.6s
285:    learn: 0.0837591    total: 16.6s    remaining: 41.5s
286:    learn: 0.0835672    total: 16.7s    remaining: 41.5s
287:    learn: 0.0834055    total: 16.8s    remaining: 41.5s
288:    learn: 0.0832002    total: 16.9s    remaining: 41.5s
289:    learn: 0.0829168    total: 17s    remaining: 41.5s
290:    learn: 0.0826480    total: 17s    remaining: 41.5s
291:    learn: 0.0823963    total: 17.1s    remaining: 41.5s
292:    learn: 0.0821079    total: 17.2s    remaining: 41.5s
293:    learn: 0.0817978    total: 17.3s    remaining: 41.5s
294:    learn: 0.0816177    total: 17.3s    remaining: 41.5s
295:    learn: 0.0813600    total: 17.4s    remaining: 41.4s
296:    learn: 0.0812284    total: 17.5s    remaining: 41.4s
297:    learn: 0.0810590    total: 17.6s    remaining: 41.4s
298:    learn: 0.0807188    total: 17.7s    remaining: 41.4s
299:    learn: 0.0805831    total: 17.8s    remaining: 41.4s
300:    learn: 0.0802811    total: 17.8s    remaining: 41.4s
301:    learn: 0.0801180    total: 17.9s    remaining: 41.4s
302:    learn: 0.0800344    total: 18s    remaining: 41.4s
303:    learn: 0.0799437    total: 18.1s    remaining: 41.4s
304:    learn: 0.0795872    total: 18.2s    remaining: 41.4s
305:    learn: 0.0793887    total: 18.2s    remaining: 41.4s
306:    learn: 0.0792553    total: 18.3s    remaining: 41.3s
307:    learn: 0.0790550    total: 18.4s    remaining: 41.3s
308:    learn: 0.0788949    total: 18.5s    remaining: 41.3s
309:    learn: 0.0787694    total: 18.6s    remaining: 41.3s
310:    learn: 0.0786095    total: 18.6s    remaining: 41.3s
311:    learn: 0.0783436    total: 18.7s    remaining: 41.3s
312:    learn: 0.0781246    total: 18.8s    remaining: 41.3s
313:    learn: 0.0780335    total: 18.9s    remaining: 41.3s
314:    learn: 0.0778421    total: 19s    remaining: 41.3s
315:    learn: 0.0776884    total: 19.1s    remaining: 41.3s
316:    learn: 0.0774921    total: 19.1s    remaining: 41.2s
317:    learn: 0.0773268    total: 19.2s    remaining: 41.2s
318:    learn: 0.0771909    total: 19.3s    remaining: 41.1s
319:    learn: 0.0771140    total: 19.3s    remaining: 41.1s
320:    learn: 0.0769700    total: 19.4s    remaining: 41.1s
321:    learn: 0.0767145    total: 19.5s    remaining: 41s
322:    learn: 0.0765796    total: 19.6s    remaining: 41s
323:    learn: 0.0764386    total: 19.7s    remaining: 41s
324:    learn: 0.0763118    total: 19.7s    remaining: 41s
325:    learn: 0.0760452    total: 19.8s    remaining: 41s
326:    learn: 0.0759214    total: 19.9s    remaining: 41s
327:    learn: 0.0758953    total: 19.9s    remaining: 40.8s
328:    learn: 0.0758316    total: 19.9s    remaining: 40.6s
329:    learn: 0.0755765    total: 20s    remaining: 40.6s
330:    learn: 0.0754392    total: 20.1s    remaining: 40.7s
331:    learn: 0.0753975    total: 20.1s    remaining: 40.5s
332:    learn: 0.0752450    total: 20.2s    remaining: 40.5s
333:    learn: 0.0750084    total: 20.3s    remaining: 40.4s
334:    learn: 0.0747665    total: 20.4s    remaining: 40.4s
335:    learn: 0.0746458    total: 20.4s    remaining: 40.4s
336:    learn: 0.0744223    total: 20.5s    remaining: 40.4s
337:    learn: 0.0742375    total: 20.6s    remaining: 40.4s
338:    learn: 0.0741206    total: 20.7s    remaining: 40.3s
339:    learn: 0.0738751    total: 20.8s    remaining: 40.3s
340:    learn: 0.0738069    total: 20.8s    remaining: 40.3s
341:    learn: 0.0736982    total: 20.9s    remaining: 40.3s
342:    learn: 0.0735798    total: 21s    remaining: 40.2s
343:    learn: 0.0733606    total: 21.1s    remaining: 40.2s
344:    learn: 0.0731264    total: 21.2s    remaining: 40.2s
345:    learn: 0.0729077    total: 21.2s    remaining: 40.1s
346:    learn: 0.0727358    total: 21.3s    remaining: 40s
347:    learn: 0.0726109    total: 21.4s    remaining: 40s
348:    learn: 0.0723158    total: 21.4s    remaining: 40s
349:    learn: 0.0721597    total: 21.5s    remaining: 39.9s
350:    learn: 0.0720101    total: 21.6s    remaining: 39.9s
351:    learn: 0.0718753    total: 21.7s    remaining: 39.9s
352:    learn: 0.0718178    total: 21.7s    remaining: 39.8s
353:    learn: 0.0716506    total: 21.8s    remaining: 39.8s
354:    learn: 0.0713943    total: 21.9s    remaining: 39.8s
355:    learn: 0.0711916    total: 22s    remaining: 39.7s
356:    learn: 0.0710649    total: 22.1s    remaining: 39.7s
357:    learn: 0.0709570    total: 22.1s    remaining: 39.7s
358:    learn: 0.0709018    total: 22.2s    remaining: 39.7s
359:    learn: 0.0708145    total: 22.3s    remaining: 39.6s
360:    learn: 0.0707875    total: 22.4s    remaining: 39.6s
361:    learn: 0.0706960    total: 22.5s    remaining: 39.6s
362:    learn: 0.0704835    total: 22.5s    remaining: 39.6s
363:    learn: 0.0704660    total: 22.6s    remaining: 39.4s
364:    learn: 0.0704422    total: 22.6s    remaining: 39.3s
365:    learn: 0.0703499    total: 22.7s    remaining: 39.3s
366:    learn: 0.0702633    total: 22.8s    remaining: 39.2s
367:    learn: 0.0702122    total: 22.8s    remaining: 39.2s
368:    learn: 0.0700797    total: 22.9s    remaining: 39.1s
369:    learn: 0.0699954    total: 23s    remaining: 39.1s
370:    learn: 0.0698622    total: 23s    remaining: 39.1s
371:    learn: 0.0697984    total: 23.1s    remaining: 39s
372:    learn: 0.0697504    total: 23.2s    remaining: 39s
373:    learn: 0.0696325    total: 23.3s    remaining: 39s
374:    learn: 0.0695827    total: 23.3s    remaining: 38.8s
375:    learn: 0.0694660    total: 23.4s    remaining: 38.8s
376:    learn: 0.0693729    total: 23.4s    remaining: 38.8s
377:    learn: 0.0691513    total: 23.5s    remaining: 38.7s
378:    learn: 0.0690860    total: 23.6s    remaining: 38.6s
379:    learn: 0.0689910    total: 23.7s    remaining: 38.6s
380:    learn: 0.0688406    total: 23.7s    remaining: 38.6s
381:    learn: 0.0687428    total: 23.8s    remaining: 38.5s
382:    learn: 0.0686822    total: 23.9s    remaining: 38.5s
383:    learn: 0.0685927    total: 24s    remaining: 38.5s
384:    learn: 0.0684202    total: 24.1s    remaining: 38.4s
385:    learn: 0.0682822    total: 24.1s    remaining: 38.4s
386:    learn: 0.0682056    total: 24.2s    remaining: 38.4s
387:    learn: 0.0681720    total: 24.2s    remaining: 38.2s
388:    learn: 0.0680716    total: 24.3s    remaining: 38.2s
389:    learn: 0.0679639    total: 24.4s    remaining: 38.2s
390:    learn: 0.0677765    total: 24.5s    remaining: 38.2s
391:    learn: 0.0677630    total: 24.5s    remaining: 38s
392:    learn: 0.0676595    total: 24.6s    remaining: 38s
393:    learn: 0.0675553    total: 24.7s    remaining: 38s
394:    learn: 0.0674711    total: 24.8s    remaining: 37.9s
395:    learn: 0.0673370    total: 24.8s    remaining: 37.9s
396:    learn: 0.0673117    total: 24.9s    remaining: 37.8s
397:    learn: 0.0672352    total: 24.9s    remaining: 37.7s
398:    learn: 0.0671287    total: 25s    remaining: 37.7s
399:    learn: 0.0670080    total: 25.1s    remaining: 37.6s
400:    learn: 0.0668827    total: 25.2s    remaining: 37.6s
401:    learn: 0.0667930    total: 25.3s    remaining: 37.6s
402:    learn: 0.0667226    total: 25.3s    remaining: 37.5s
403:    learn: 0.0666502    total: 25.4s    remaining: 37.5s
404:    learn: 0.0665833    total: 25.5s    remaining: 37.4s
405:    learn: 0.0664556    total: 25.6s    remaining: 37.4s
406:    learn: 0.0663563    total: 25.7s    remaining: 37.4s
407:    learn: 0.0662776    total: 25.7s    remaining: 37.2s
408:    learn: 0.0662696    total: 25.7s    remaining: 37.1s
409:    learn: 0.0662331    total: 25.7s    remaining: 37s
410:    learn: 0.0662140    total: 25.8s    remaining: 36.9s
411:    learn: 0.0660842    total: 25.9s    remaining: 36.9s
412:    learn: 0.0659252    total: 25.9s    remaining: 36.9s
413:    learn: 0.0658368    total: 26s    remaining: 36.8s
414:    learn: 0.0657215    total: 26.1s    remaining: 36.8s
415:    learn: 0.0656556    total: 26.2s    remaining: 36.8s
416:    learn: 0.0655536    total: 26.3s    remaining: 36.7s
417:    learn: 0.0655416    total: 26.3s    remaining: 36.6s
418:    learn: 0.0655031    total: 26.4s    remaining: 36.5s
419:    learn: 0.0654451    total: 26.4s    remaining: 36.5s
420:    learn: 0.0653336    total: 26.5s    remaining: 36.5s
421:    learn: 0.0652475    total: 26.6s    remaining: 36.4s
422:    learn: 0.0652147    total: 26.6s    remaining: 36.3s
423:    learn: 0.0650865    total: 26.7s    remaining: 36.3s
424:    learn: 0.0649944    total: 26.8s    remaining: 36.2s
425:    learn: 0.0649487    total: 26.9s    remaining: 36.2s
426:    learn: 0.0648933    total: 26.9s    remaining: 36.2s
427:    learn: 0.0647768    total: 27s    remaining: 36.1s
428:    learn: 0.0646220    total: 27.1s    remaining: 36.1s
429:    learn: 0.0645673    total: 27.2s    remaining: 36s
430:    learn: 0.0644823    total: 27.3s    remaining: 36s
431:    learn: 0.0644046    total: 27.4s    remaining: 36s
432:    learn: 0.0643120    total: 27.4s    remaining: 35.9s
433:    learn: 0.0642234    total: 27.5s    remaining: 35.9s
434:    learn: 0.0641458    total: 27.6s    remaining: 35.8s
435:    learn: 0.0640602    total: 27.7s    remaining: 35.8s
436:    learn: 0.0639917    total: 27.8s    remaining: 35.8s
437:    learn: 0.0638438    total: 27.8s    remaining: 35.7s
438:    learn: 0.0637128    total: 27.9s    remaining: 35.7s
439:    learn: 0.0636231    total: 28s    remaining: 35.7s
440:    learn: 0.0635662    total: 28.1s    remaining: 35.6s
441:    learn: 0.0634986    total: 28.2s    remaining: 35.6s
442:    learn: 0.0634137    total: 28.3s    remaining: 35.5s
443:    learn: 0.0633724    total: 28.3s    remaining: 35.5s
444:    learn: 0.0632873    total: 28.4s    remaining: 35.4s
445:    learn: 0.0632346    total: 28.5s    remaining: 35.4s
446:    learn: 0.0631867    total: 28.6s    remaining: 35.3s
447:    learn: 0.0631247    total: 28.7s    remaining: 35.3s
448:    learn: 0.0630819    total: 28.7s    remaining: 35.3s
449:    learn: 0.0630281    total: 28.8s    remaining: 35.2s
450:    learn: 0.0629809    total: 28.9s    remaining: 35.2s
451:    learn: 0.0629226    total: 29s    remaining: 35.1s
452:    learn: 0.0628627    total: 29.1s    remaining: 35.1s
453:    learn: 0.0627670    total: 29.1s    remaining: 35s
454:    learn: 0.0627075    total: 29.2s    remaining: 35s
455:    learn: 0.0626426    total: 29.3s    remaining: 35s
456:    learn: 0.0625294    total: 29.4s    remaining: 34.9s
457:    learn: 0.0624632    total: 29.5s    remaining: 34.9s
458:    learn: 0.0623462    total: 29.5s    remaining: 34.8s
459:    learn: 0.0622866    total: 29.6s    remaining: 34.8s
460:    learn: 0.0622428    total: 29.7s    remaining: 34.7s
461:    learn: 0.0622024    total: 29.8s    remaining: 34.7s
462:    learn: 0.0621693    total: 29.9s    remaining: 34.6s
463:    learn: 0.0620835    total: 29.9s    remaining: 34.6s
464:    learn: 0.0620437    total: 30s    remaining: 34.5s
465:    learn: 0.0620116    total: 30s    remaining: 34.4s
466:    learn: 0.0619525    total: 30.1s    remaining: 34.4s
467:    learn: 0.0619261    total: 30.2s    remaining: 34.3s
468:    learn: 0.0618856    total: 30.3s    remaining: 34.3s
469:    learn: 0.0618618    total: 30.4s    remaining: 34.3s
470:    learn: 0.0617853    total: 30.5s    remaining: 34.2s
471:    learn: 0.0616711    total: 30.6s    remaining: 34.2s
472:    learn: 0.0616231    total: 30.6s    remaining: 34.1s
473:    learn: 0.0615676    total: 30.7s    remaining: 34s
474:    learn: 0.0614766    total: 30.8s    remaining: 34s
475:    learn: 0.0614160    total: 30.9s    remaining: 34s
476:    learn: 0.0613391    total: 30.9s    remaining: 33.9s
477:    learn: 0.0612957    total: 31s    remaining: 33.9s
478:    learn: 0.0612656    total: 31.1s    remaining: 33.8s
479:    learn: 0.0611741    total: 31.2s    remaining: 33.8s
480:    learn: 0.0611504    total: 31.2s    remaining: 33.7s
481:    learn: 0.0610432    total: 31.3s    remaining: 33.7s
482:    learn: 0.0609883    total: 31.4s    remaining: 33.6s
483:    learn: 0.0608631    total: 31.5s    remaining: 33.6s
484:    learn: 0.0608306    total: 31.6s    remaining: 33.5s
485:    learn: 0.0607123    total: 31.6s    remaining: 33.5s
486:    learn: 0.0606671    total: 31.7s    remaining: 33.4s
487:    learn: 0.0605979    total: 31.8s    remaining: 33.4s
488:    learn: 0.0604872    total: 31.9s    remaining: 33.3s
489:    learn: 0.0603922    total: 32s    remaining: 33.3s
490:    learn: 0.0602644    total: 32s    remaining: 33.2s
491:    learn: 0.0601780    total: 32.1s    remaining: 33.2s
492:    learn: 0.0601248    total: 32.2s    remaining: 33.1s
493:    learn: 0.0600746    total: 32.3s    remaining: 33.1s
494:    learn: 0.0600219    total: 32.4s    remaining: 33s
495:    learn: 0.0599442    total: 32.5s    remaining: 33s
496:    learn: 0.0598624    total: 32.5s    remaining: 32.9s
497:    learn: 0.0598182    total: 32.6s    remaining: 32.9s
498:    learn: 0.0597954    total: 32.7s    remaining: 32.8s
499:    learn: 0.0597308    total: 32.8s    remaining: 32.8s
500:    learn: 0.0596585    total: 32.9s    remaining: 32.7s
501:    learn: 0.0596202    total: 32.9s    remaining: 32.7s
502:    learn: 0.0595124    total: 33s    remaining: 32.6s
503:    learn: 0.0594510    total: 33.1s    remaining: 32.6s
504:    learn: 0.0593790    total: 33.2s    remaining: 32.5s
505:    learn: 0.0593038    total: 33.3s    remaining: 32.5s
506:    learn: 0.0591878    total: 33.3s    remaining: 32.4s
507:    learn: 0.0591315    total: 33.4s    remaining: 32.4s
508:    learn: 0.0590626    total: 33.5s    remaining: 32.3s
509:    learn: 0.0589951    total: 33.6s    remaining: 32.3s
510:    learn: 0.0589847    total: 33.7s    remaining: 32.2s
511:    learn: 0.0589442    total: 33.7s    remaining: 32.2s
512:    learn: 0.0589154    total: 33.8s    remaining: 32.1s
513:    learn: 0.0588723    total: 33.9s    remaining: 32.1s
514:    learn: 0.0588471    total: 34s    remaining: 32s
515:    learn: 0.0588147    total: 34.1s    remaining: 31.9s
516:    learn: 0.0587502    total: 34.1s    remaining: 31.9s
517:    learn: 0.0587225    total: 34.2s    remaining: 31.8s
518:    learn: 0.0586514    total: 34.3s    remaining: 31.8s
519:    learn: 0.0585996    total: 34.4s    remaining: 31.7s
520:    learn: 0.0585270    total: 34.5s    remaining: 31.7s
521:    learn: 0.0584892    total: 34.5s    remaining: 31.6s
522:    learn: 0.0584533    total: 34.6s    remaining: 31.6s
523:    learn: 0.0583926    total: 34.7s    remaining: 31.5s
524:    learn: 0.0583423    total: 34.8s    remaining: 31.5s
525:    learn: 0.0582639    total: 34.9s    remaining: 31.4s
526:    learn: 0.0582035    total: 34.9s    remaining: 31.4s
527:    learn: 0.0581350    total: 35s    remaining: 31.3s
528:    learn: 0.0581145    total: 35.1s    remaining: 31.3s
529:    learn: 0.0580438    total: 35.2s    remaining: 31.2s
530:    learn: 0.0580047    total: 35.3s    remaining: 31.1s
531:    learn: 0.0579979    total: 35.3s    remaining: 31s
532:    learn: 0.0579513    total: 35.4s    remaining: 31s
533:    learn: 0.0578789    total: 35.4s    remaining: 30.9s
534:    learn: 0.0578597    total: 35.5s    remaining: 30.9s
535:    learn: 0.0578236    total: 35.6s    remaining: 30.8s
536:    learn: 0.0577576    total: 35.6s    remaining: 30.7s
537:    learn: 0.0577555    total: 35.6s    remaining: 30.6s
538:    learn: 0.0576661    total: 35.7s    remaining: 30.6s
539:    learn: 0.0576423    total: 35.8s    remaining: 30.5s
540:    learn: 0.0576112    total: 35.9s    remaining: 30.5s
541:    learn: 0.0576056    total: 35.9s    remaining: 30.3s
542:    learn: 0.0575204    total: 36s    remaining: 30.3s
543:    learn: 0.0574944    total: 36.1s    remaining: 30.2s
544:    learn: 0.0574624    total: 36.1s    remaining: 30.2s
545:    learn: 0.0574339    total: 36.2s    remaining: 30.1s
546:    learn: 0.0574113    total: 36.2s    remaining: 30s
547:    learn: 0.0573514    total: 36.3s    remaining: 30s
548:    learn: 0.0572865    total: 36.4s    remaining: 29.9s
549:    learn: 0.0572687    total: 36.5s    remaining: 29.8s
550:    learn: 0.0572307    total: 36.6s    remaining: 29.8s
551:    learn: 0.0571639    total: 36.6s    remaining: 29.7s
552:    learn: 0.0571003    total: 36.7s    remaining: 29.7s
553:    learn: 0.0570416    total: 36.8s    remaining: 29.6s
554:    learn: 0.0570358    total: 36.8s    remaining: 29.5s
555:    learn: 0.0570145    total: 36.9s    remaining: 29.5s
556:    learn: 0.0570025    total: 37s    remaining: 29.4s
557:    learn: 0.0569711    total: 37s    remaining: 29.3s
558:    learn: 0.0569280    total: 37.1s    remaining: 29.2s
559:    learn: 0.0568576    total: 37.2s    remaining: 29.2s
560:    learn: 0.0567969    total: 37.2s    remaining: 29.1s
561:    learn: 0.0567652    total: 37.2s    remaining: 29s
562:    learn: 0.0567616    total: 37.3s    remaining: 28.9s
563:    learn: 0.0567243    total: 37.3s    remaining: 28.9s
564:    learn: 0.0566695    total: 37.4s    remaining: 28.8s
565:    learn: 0.0566430    total: 37.5s    remaining: 28.8s
566:    learn: 0.0566045    total: 37.6s    remaining: 28.7s
567:    learn: 0.0566044    total: 37.6s    remaining: 28.6s
568:    learn: 0.0566016    total: 37.6s    remaining: 28.5s
569:    learn: 0.0565509    total: 37.7s    remaining: 28.4s
570:    learn: 0.0565482    total: 37.7s    remaining: 28.3s
571:    learn: 0.0564908    total: 37.8s    remaining: 28.2s
572:    learn: 0.0564157    total: 37.8s    remaining: 28.2s
573:    learn: 0.0563966    total: 37.9s    remaining: 28.1s
574:    learn: 0.0563267    total: 38s    remaining: 28.1s
575:    learn: 0.0563010    total: 38.1s    remaining: 28s
576:    learn: 0.0562972    total: 38.1s    remaining: 27.9s
577:    learn: 0.0562788    total: 38.1s    remaining: 27.8s
578:    learn: 0.0562703    total: 38.1s    remaining: 27.7s
579:    learn: 0.0562224    total: 38.2s    remaining: 27.6s
580:    learn: 0.0562128    total: 38.2s    remaining: 27.5s
581:    learn: 0.0561442    total: 38.3s    remaining: 27.5s
582:    learn: 0.0561258    total: 38.3s    remaining: 27.4s
583:    learn: 0.0560771    total: 38.4s    remaining: 27.4s
584:    learn: 0.0560251    total: 38.5s    remaining: 27.3s
585:    learn: 0.0560085    total: 38.6s    remaining: 27.3s
586:    learn: 0.0559557    total: 38.7s    remaining: 27.2s
587:    learn: 0.0559067    total: 38.7s    remaining: 27.1s
588:    learn: 0.0558655    total: 38.8s    remaining: 27.1s
589:    learn: 0.0558199    total: 38.9s    remaining: 27s
590:    learn: 0.0557767    total: 39s    remaining: 27s
591:    learn: 0.0557538    total: 39.1s    remaining: 26.9s
592:    learn: 0.0557506    total: 39.1s    remaining: 26.8s
593:    learn: 0.0557241    total: 39.2s    remaining: 26.8s
594:    learn: 0.0556955    total: 39.2s    remaining: 26.7s
595:    learn: 0.0556704    total: 39.3s    remaining: 26.6s
596:    learn: 0.0556488    total: 39.4s    remaining: 26.6s
597:    learn: 0.0556369    total: 39.5s    remaining: 26.5s
598:    learn: 0.0556082    total: 39.6s    remaining: 26.5s
599:    learn: 0.0555476    total: 39.6s    remaining: 26.4s
600:    learn: 0.0555254    total: 39.7s    remaining: 26.3s
601:    learn: 0.0555042    total: 39.7s    remaining: 26.3s
602:    learn: 0.0554939    total: 39.8s    remaining: 26.2s
603:    learn: 0.0554583    total: 39.9s    remaining: 26.2s
604:    learn: 0.0554121    total: 40s    remaining: 26.1s
605:    learn: 0.0553839    total: 40.1s    remaining: 26s
606:    learn: 0.0553179    total: 40.1s    remaining: 26s
607:    learn: 0.0552890    total: 40.2s    remaining: 25.9s
608:    learn: 0.0552693    total: 40.3s    remaining: 25.9s
609:    learn: 0.0552215    total: 40.4s    remaining: 25.8s
610:    learn: 0.0551917    total: 40.5s    remaining: 25.8s
611:    learn: 0.0551509    total: 40.6s    remaining: 25.7s
612:    learn: 0.0551163    total: 40.6s    remaining: 25.7s
613:    learn: 0.0550651    total: 40.7s    remaining: 25.6s
614:    learn: 0.0549956    total: 40.8s    remaining: 25.5s
615:    learn: 0.0549358    total: 40.9s    remaining: 25.5s
616:    learn: 0.0548813    total: 40.9s    remaining: 25.4s
617:    learn: 0.0548195    total: 41s    remaining: 25.3s
618:    learn: 0.0547734    total: 41.1s    remaining: 25.3s
619:    learn: 0.0547143    total: 41.1s    remaining: 25.2s
620:    learn: 0.0546948    total: 41.2s    remaining: 25.2s
621:    learn: 0.0546472    total: 41.3s    remaining: 25.1s
622:    learn: 0.0546137    total: 41.4s    remaining: 25s
623:    learn: 0.0545772    total: 41.5s    remaining: 25s
624:    learn: 0.0545610    total: 41.5s    remaining: 24.9s
625:    learn: 0.0545310    total: 41.6s    remaining: 24.9s
626:    learn: 0.0544733    total: 41.7s    remaining: 24.8s
627:    learn: 0.0544322    total: 41.8s    remaining: 24.8s
628:    learn: 0.0543813    total: 41.9s    remaining: 24.7s
629:    learn: 0.0543221    total: 42s    remaining: 24.6s
630:    learn: 0.0542785    total: 42s    remaining: 24.6s
631:    learn: 0.0542340    total: 42.1s    remaining: 24.5s
632:    learn: 0.0542043    total: 42.2s    remaining: 24.5s
633:    learn: 0.0541720    total: 42.3s    remaining: 24.4s
634:    learn: 0.0541348    total: 42.4s    remaining: 24.4s
635:    learn: 0.0541280    total: 42.4s    remaining: 24.2s
636:    learn: 0.0540815    total: 42.4s    remaining: 24.2s
637:    learn: 0.0540410    total: 42.5s    remaining: 24.1s
638:    learn: 0.0540121    total: 42.6s    remaining: 24.1s
639:    learn: 0.0539801    total: 42.7s    remaining: 24s
640:    learn: 0.0539751    total: 42.7s    remaining: 23.9s
641:    learn: 0.0539553    total: 42.8s    remaining: 23.9s
642:    learn: 0.0539153    total: 42.9s    remaining: 23.8s
643:    learn: 0.0538733    total: 42.9s    remaining: 23.7s
644:    learn: 0.0538325    total: 43s    remaining: 23.7s
645:    learn: 0.0538109    total: 43.1s    remaining: 23.6s
646:    learn: 0.0537777    total: 43.2s    remaining: 23.6s
647:    learn: 0.0537541    total: 43.3s    remaining: 23.5s
648:    learn: 0.0537379    total: 43.3s    remaining: 23.4s
649:    learn: 0.0536988    total: 43.4s    remaining: 23.4s
650:    learn: 0.0536781    total: 43.5s    remaining: 23.3s
651:    learn: 0.0536645    total: 43.5s    remaining: 23.2s
652:    learn: 0.0536346    total: 43.6s    remaining: 23.2s
653:    learn: 0.0536225    total: 43.7s    remaining: 23.1s
654:    learn: 0.0535963    total: 43.8s    remaining: 23s
655:    learn: 0.0535883    total: 43.8s    remaining: 23s
656:    learn: 0.0535883    total: 43.8s    remaining: 22.9s
657:    learn: 0.0535780    total: 43.9s    remaining: 22.8s
658:    learn: 0.0535419    total: 44s    remaining: 22.8s
659:    learn: 0.0535116    total: 44.1s    remaining: 22.7s
660:    learn: 0.0534520    total: 44.2s    remaining: 22.7s
661:    learn: 0.0534199    total: 44.3s    remaining: 22.6s
662:    learn: 0.0533850    total: 44.3s    remaining: 22.5s
663:    learn: 0.0533775    total: 44.4s    remaining: 22.5s
664:    learn: 0.0533391    total: 44.5s    remaining: 22.4s
665:    learn: 0.0533314    total: 44.5s    remaining: 22.3s
666:    learn: 0.0532823    total: 44.6s    remaining: 22.3s
667:    learn: 0.0532240    total: 44.7s    remaining: 22.2s
668:    learn: 0.0532200    total: 44.7s    remaining: 22.1s
669:    learn: 0.0531951    total: 44.7s    remaining: 22s
670:    learn: 0.0531700    total: 44.8s    remaining: 21.9s
671:    learn: 0.0531532    total: 44.8s    remaining: 21.9s
672:    learn: 0.0531407    total: 44.9s    remaining: 21.8s
673:    learn: 0.0531192    total: 45s    remaining: 21.8s
674:    learn: 0.0530832    total: 45.1s    remaining: 21.7s
675:    learn: 0.0530787    total: 45.1s    remaining: 21.6s
676:    learn: 0.0530344    total: 45.2s    remaining: 21.5s
677:    learn: 0.0529996    total: 45.2s    remaining: 21.5s
678:    learn: 0.0529649    total: 45.3s    remaining: 21.4s
679:    learn: 0.0529292    total: 45.4s    remaining: 21.4s
680:    learn: 0.0529233    total: 45.4s    remaining: 21.3s
681:    learn: 0.0528912    total: 45.5s    remaining: 21.2s
682:    learn: 0.0528811    total: 45.6s    remaining: 21.2s
683:    learn: 0.0528611    total: 45.7s    remaining: 21.1s
684:    learn: 0.0528421    total: 45.8s    remaining: 21.1s
685:    learn: 0.0528292    total: 45.9s    remaining: 21s
686:    learn: 0.0528068    total: 45.9s    remaining: 20.9s
687:    learn: 0.0527948    total: 46s    remaining: 20.9s
688:    learn: 0.0527652    total: 46.1s    remaining: 20.8s
689:    learn: 0.0527163    total: 46.2s    remaining: 20.7s
690:    learn: 0.0526741    total: 46.3s    remaining: 20.7s
691:    learn: 0.0526589    total: 46.3s    remaining: 20.6s
692:    learn: 0.0526302    total: 46.4s    remaining: 20.6s
693:    learn: 0.0526016    total: 46.5s    remaining: 20.5s
694:    learn: 0.0525761    total: 46.6s    remaining: 20.4s
695:    learn: 0.0525395    total: 46.7s    remaining: 20.4s
696:    learn: 0.0525001    total: 46.7s    remaining: 20.3s
697:    learn: 0.0524684    total: 46.8s    remaining: 20.3s
698:    learn: 0.0524328    total: 46.9s    remaining: 20.2s
699:    learn: 0.0524017    total: 47s    remaining: 20.1s
700:    learn: 0.0523965    total: 47s    remaining: 20s
701:    learn: 0.0523931    total: 47.1s    remaining: 20s
702:    learn: 0.0523444    total: 47.1s    remaining: 19.9s
703:    learn: 0.0523082    total: 47.2s    remaining: 19.9s
704:    learn: 0.0523046    total: 47.2s    remaining: 19.8s
705:    learn: 0.0522693    total: 47.3s    remaining: 19.7s
706:    learn: 0.0522304    total: 47.4s    remaining: 19.6s
707:    learn: 0.0521832    total: 47.5s    remaining: 19.6s
708:    learn: 0.0521677    total: 47.5s    remaining: 19.5s
709:    learn: 0.0521393    total: 47.6s    remaining: 19.5s
710:    learn: 0.0521192    total: 47.7s    remaining: 19.4s
711:    learn: 0.0520910    total: 47.8s    remaining: 19.3s
712:    learn: 0.0520517    total: 47.9s    remaining: 19.3s
713:    learn: 0.0520207    total: 47.9s    remaining: 19.2s
714:    learn: 0.0519936    total: 48s    remaining: 19.1s
715:    learn: 0.0519737    total: 48.1s    remaining: 19.1s
716:    learn: 0.0519543    total: 48.2s    remaining: 19s
717:    learn: 0.0519059    total: 48.3s    remaining: 19s
718:    learn: 0.0519015    total: 48.3s    remaining: 18.9s
719:    learn: 0.0518879    total: 48.4s    remaining: 18.8s
720:    learn: 0.0518622    total: 48.5s    remaining: 18.8s
721:    learn: 0.0518434    total: 48.5s    remaining: 18.7s
722:    learn: 0.0518251    total: 48.6s    remaining: 18.6s
723:    learn: 0.0518006    total: 48.7s    remaining: 18.6s
724:    learn: 0.0517861    total: 48.8s    remaining: 18.5s
725:    learn: 0.0517640    total: 48.9s    remaining: 18.4s
726:    learn: 0.0517639    total: 48.9s    remaining: 18.4s
727:    learn: 0.0517346    total: 49s    remaining: 18.3s
728:    learn: 0.0517253    total: 49s    remaining: 18.2s
729:    learn: 0.0516807    total: 49.1s    remaining: 18.1s
730:    learn: 0.0516806    total: 49.1s    remaining: 18.1s
731:    learn: 0.0516739    total: 49.1s    remaining: 18s
732:    learn: 0.0516491    total: 49.2s    remaining: 17.9s
733:    learn: 0.0516491    total: 49.2s    remaining: 17.8s
734:    learn: 0.0515986    total: 49.3s    remaining: 17.8s
735:    learn: 0.0515787    total: 49.3s    remaining: 17.7s
736:    learn: 0.0515602    total: 49.4s    remaining: 17.6s
737:    learn: 0.0515559    total: 49.5s    remaining: 17.6s
738:    learn: 0.0515476    total: 49.5s    remaining: 17.5s
739:    learn: 0.0515234    total: 49.6s    remaining: 17.4s
740:    learn: 0.0514953    total: 49.7s    remaining: 17.4s
741:    learn: 0.0514785    total: 49.8s    remaining: 17.3s
742:    learn: 0.0514769    total: 49.8s    remaining: 17.2s
743:    learn: 0.0514759    total: 49.8s    remaining: 17.1s
744:    learn: 0.0514683    total: 49.8s    remaining: 17s
745:    learn: 0.0514682    total: 49.8s    remaining: 16.9s
746:    learn: 0.0514407    total: 49.9s    remaining: 16.9s
747:    learn: 0.0514155    total: 49.9s    remaining: 16.8s
748:    learn: 0.0513860    total: 50s    remaining: 16.7s
749:    learn: 0.0513621    total: 50s    remaining: 16.7s
750:    learn: 0.0513435    total: 50.1s    remaining: 16.6s
751:    learn: 0.0513006    total: 50.2s    remaining: 16.6s
752:    learn: 0.0512598    total: 50.3s    remaining: 16.5s
753:    learn: 0.0512391    total: 50.4s    remaining: 16.4s
754:    learn: 0.0512084    total: 50.5s    remaining: 16.4s
755:    learn: 0.0512066    total: 50.5s    remaining: 16.3s
756:    learn: 0.0511812    total: 50.5s    remaining: 16.2s
757:    learn: 0.0511776    total: 50.5s    remaining: 16.1s
758:    learn: 0.0511644    total: 50.6s    remaining: 16.1s
759:    learn: 0.0511427    total: 50.7s    remaining: 16s
760:    learn: 0.0511171    total: 50.8s    remaining: 15.9s
761:    learn: 0.0510823    total: 50.8s    remaining: 15.9s
762:    learn: 0.0510748    total: 50.9s    remaining: 15.8s
763:    learn: 0.0510643    total: 50.9s    remaining: 15.7s
764:    learn: 0.0510521    total: 51s    remaining: 15.7s
765:    learn: 0.0510316    total: 51s    remaining: 15.6s
766:    learn: 0.0510176    total: 51.1s    remaining: 15.5s
767:    learn: 0.0510026    total: 51.2s    remaining: 15.5s
768:    learn: 0.0509902    total: 51.3s    remaining: 15.4s
769:    learn: 0.0509799    total: 51.4s    remaining: 15.3s
770:    learn: 0.0509573    total: 51.4s    remaining: 15.3s
771:    learn: 0.0509404    total: 51.5s    remaining: 15.2s
772:    learn: 0.0509252    total: 51.6s    remaining: 15.2s
773:    learn: 0.0509216    total: 51.7s    remaining: 15.1s
774:    learn: 0.0508946    total: 51.8s    remaining: 15s
775:    learn: 0.0508798    total: 51.8s    remaining: 15s
776:    learn: 0.0508618    total: 51.9s    remaining: 14.9s
777:    learn: 0.0508549    total: 52s    remaining: 14.8s
778:    learn: 0.0508241    total: 52s    remaining: 14.8s
779:    learn: 0.0508036    total: 52.1s    remaining: 14.7s
780:    learn: 0.0507911    total: 52.2s    remaining: 14.6s
781:    learn: 0.0507803    total: 52.3s    remaining: 14.6s
782:    learn: 0.0507605    total: 52.4s    remaining: 14.5s
783:    learn: 0.0507604    total: 52.4s    remaining: 14.4s
784:    learn: 0.0507474    total: 52.5s    remaining: 14.4s
785:    learn: 0.0507139    total: 52.5s    remaining: 14.3s
786:    learn: 0.0507050    total: 52.6s    remaining: 14.2s
787:    learn: 0.0506937    total: 52.7s    remaining: 14.2s
788:    learn: 0.0506593    total: 52.8s    remaining: 14.1s
789:    learn: 0.0506356    total: 52.9s    remaining: 14s
790:    learn: 0.0506036    total: 52.9s    remaining: 14s
791:    learn: 0.0505768    total: 53s    remaining: 13.9s
792:    learn: 0.0505455    total: 53.1s    remaining: 13.9s
793:    learn: 0.0505455    total: 53.1s    remaining: 13.8s
794:    learn: 0.0505376    total: 53.2s    remaining: 13.7s
795:    learn: 0.0505344    total: 53.2s    remaining: 13.6s
796:    learn: 0.0505222    total: 53.3s    remaining: 13.6s
797:    learn: 0.0505047    total: 53.4s    remaining: 13.5s
798:    learn: 0.0504874    total: 53.4s    remaining: 13.4s
799:    learn: 0.0504531    total: 53.5s    remaining: 13.4s
800:    learn: 0.0504323    total: 53.6s    remaining: 13.3s
801:    learn: 0.0504025    total: 53.7s    remaining: 13.3s
802:    learn: 0.0503826    total: 53.8s    remaining: 13.2s
803:    learn: 0.0503826    total: 53.8s    remaining: 13.1s
804:    learn: 0.0503747    total: 53.9s    remaining: 13s
805:    learn: 0.0503445    total: 53.9s    remaining: 13s
806:    learn: 0.0503168    total: 54s    remaining: 12.9s
807:    learn: 0.0502967    total: 54.1s    remaining: 12.9s
808:    learn: 0.0502850    total: 54.2s    remaining: 12.8s
809:    learn: 0.0502785    total: 54.2s    remaining: 12.7s
810:    learn: 0.0502558    total: 54.3s    remaining: 12.7s
811:    learn: 0.0502286    total: 54.4s    remaining: 12.6s
812:    learn: 0.0502024    total: 54.5s    remaining: 12.5s
813:    learn: 0.0501775    total: 54.5s    remaining: 12.5s
814:    learn: 0.0501774    total: 54.5s    remaining: 12.4s
815:    learn: 0.0501544    total: 54.6s    remaining: 12.3s
816:    learn: 0.0501311    total: 54.7s    remaining: 12.2s
817:    learn: 0.0501118    total: 54.8s    remaining: 12.2s
818:    learn: 0.0500872    total: 54.8s    remaining: 12.1s
819:    learn: 0.0500770    total: 54.9s    remaining: 12.1s
820:    learn: 0.0500710    total: 55s    remaining: 12s
821:    learn: 0.0500498    total: 55.1s    remaining: 11.9s
822:    learn: 0.0500335    total: 55.2s    remaining: 11.9s
823:    learn: 0.0500208    total: 55.2s    remaining: 11.8s
824:    learn: 0.0500152    total: 55.3s    remaining: 11.7s
825:    learn: 0.0500075    total: 55.4s    remaining: 11.7s
826:    learn: 0.0500043    total: 55.5s    remaining: 11.6s
827:    learn: 0.0499862    total: 55.6s    remaining: 11.5s
828:    learn: 0.0499706    total: 55.6s    remaining: 11.5s
829:    learn: 0.0499416    total: 55.7s    remaining: 11.4s
830:    learn: 0.0499375    total: 55.8s    remaining: 11.3s
831:    learn: 0.0499345    total: 55.8s    remaining: 11.3s
832:    learn: 0.0499043    total: 55.9s    remaining: 11.2s
833:    learn: 0.0498772    total: 56s    remaining: 11.1s
834:    learn: 0.0498745    total: 56.1s    remaining: 11.1s
835:    learn: 0.0498592    total: 56.1s    remaining: 11s
836:    learn: 0.0498441    total: 56.2s    remaining: 10.9s
837:    learn: 0.0498164    total: 56.3s    remaining: 10.9s
838:    learn: 0.0498028    total: 56.4s    remaining: 10.8s
839:    learn: 0.0497924    total: 56.5s    remaining: 10.8s
840:    learn: 0.0497684    total: 56.6s    remaining: 10.7s
841:    learn: 0.0497584    total: 56.6s    remaining: 10.6s
842:    learn: 0.0497366    total: 56.7s    remaining: 10.6s
843:    learn: 0.0497284    total: 56.8s    remaining: 10.5s
844:    learn: 0.0497170    total: 56.9s    remaining: 10.4s
845:    learn: 0.0497163    total: 56.9s    remaining: 10.4s
846:    learn: 0.0496952    total: 56.9s    remaining: 10.3s
847:    learn: 0.0496952    total: 56.9s    remaining: 10.2s
848:    learn: 0.0496852    total: 57s    remaining: 10.1s
849:    learn: 0.0496637    total: 57.1s    remaining: 10.1s
850:    learn: 0.0496536    total: 57.2s    remaining: 10s
851:    learn: 0.0496440    total: 57.2s    remaining: 9.94s
852:    learn: 0.0496241    total: 57.3s    remaining: 9.88s
853:    learn: 0.0496156    total: 57.4s    remaining: 9.81s
854:    learn: 0.0496015    total: 57.5s    remaining: 9.75s
855:    learn: 0.0495849    total: 57.6s    remaining: 9.68s
856:    learn: 0.0495672    total: 57.6s    remaining: 9.62s
857:    learn: 0.0495489    total: 57.7s    remaining: 9.55s
858:    learn: 0.0495285    total: 57.8s    remaining: 9.49s
859:    learn: 0.0495165    total: 57.9s    remaining: 9.42s
860:    learn: 0.0495049    total: 58s    remaining: 9.36s
861:    learn: 0.0494951    total: 58s    remaining: 9.29s
862:    learn: 0.0494693    total: 58.1s    remaining: 9.23s
863:    learn: 0.0494637    total: 58.2s    remaining: 9.16s
864:    learn: 0.0494637    total: 58.2s    remaining: 9.08s
865:    learn: 0.0494380    total: 58.3s    remaining: 9.02s
866:    learn: 0.0494372    total: 58.4s    remaining: 8.95s
867:    learn: 0.0494135    total: 58.4s    remaining: 8.88s
868:    learn: 0.0493852    total: 58.5s    remaining: 8.81s
869:    learn: 0.0493704    total: 58.5s    remaining: 8.75s
870:    learn: 0.0493704    total: 58.5s    remaining: 8.67s
871:    learn: 0.0493493    total: 58.6s    remaining: 8.6s
872:    learn: 0.0493493    total: 58.6s    remaining: 8.53s
873:    learn: 0.0493342    total: 58.7s    remaining: 8.46s
874:    learn: 0.0493237    total: 58.7s    remaining: 8.39s
875:    learn: 0.0493067    total: 58.8s    remaining: 8.32s
876:    learn: 0.0492978    total: 58.8s    remaining: 8.25s
877:    learn: 0.0492735    total: 58.9s    remaining: 8.18s
878:    learn: 0.0492658    total: 58.9s    remaining: 8.11s
879:    learn: 0.0492395    total: 59s    remaining: 8.04s
880:    learn: 0.0492164    total: 59s    remaining: 7.97s
881:    learn: 0.0492039    total: 59.1s    remaining: 7.91s
882:    learn: 0.0491852    total: 59.2s    remaining: 7.84s
883:    learn: 0.0491852    total: 59.2s    remaining: 7.76s
884:    learn: 0.0491851    total: 59.2s    remaining: 7.69s
885:    learn: 0.0491670    total: 59.2s    remaining: 7.62s
886:    learn: 0.0491507    total: 59.3s    remaining: 7.55s
887:    learn: 0.0491381    total: 59.4s    remaining: 7.49s
888:    learn: 0.0491154    total: 59.4s    remaining: 7.42s
889:    learn: 0.0491012    total: 59.5s    remaining: 7.35s
890:    learn: 0.0490930    total: 59.5s    remaining: 7.28s
891:    learn: 0.0490792    total: 59.6s    remaining: 7.21s
892:    learn: 0.0490792    total: 59.6s    remaining: 7.14s
893:    learn: 0.0490739    total: 59.6s    remaining: 7.07s
894:    learn: 0.0490451    total: 59.7s    remaining: 7s
895:    learn: 0.0490220    total: 59.7s    remaining: 6.93s
896:    learn: 0.0490141    total: 59.7s    remaining: 6.86s
897:    learn: 0.0489974    total: 59.8s    remaining: 6.79s
898:    learn: 0.0489858    total: 59.8s    remaining: 6.72s
899:    learn: 0.0489852    total: 59.9s    remaining: 6.65s
900:    learn: 0.0489770    total: 59.9s    remaining: 6.58s
901:    learn: 0.0489587    total: 60s    remaining: 6.52s
902:    learn: 0.0489458    total: 1m    remaining: 6.45s
903:    learn: 0.0489361    total: 1m    remaining: 6.38s
904:    learn: 0.0489139    total: 1m    remaining: 6.32s
905:    learn: 0.0489041    total: 1m    remaining: 6.25s
906:    learn: 0.0489041    total: 1m    remaining: 6.18s
907:    learn: 0.0488924    total: 1m    remaining: 6.11s
908:    learn: 0.0488831    total: 1m    remaining: 6.04s
909:    learn: 0.0488636    total: 1m    remaining: 5.98s
910:    learn: 0.0488445    total: 1m    remaining: 5.91s
911:    learn: 0.0488279    total: 1m    remaining: 5.84s
912:    learn: 0.0488278    total: 1m    remaining: 5.76s
913:    learn: 0.0488112    total: 1m    remaining: 5.7s
914:    learn: 0.0487917    total: 1m    remaining: 5.63s
915:    learn: 0.0487757    total: 1m    remaining: 5.57s
916:    learn: 0.0487548    total: 1m    remaining: 5.5s
917:    learn: 0.0487503    total: 1m    remaining: 5.43s
918:    learn: 0.0487350    total: 1m    remaining: 5.36s
919:    learn: 0.0487270    total: 1m    remaining: 5.29s
920:    learn: 0.0487001    total: 1m    remaining: 5.23s
921:    learn: 0.0487001    total: 1m    remaining: 5.16s
922:    learn: 0.0486902    total: 1m    remaining: 5.08s
923:    learn: 0.0486902    total: 1m    remaining: 5.01s
924:    learn: 0.0486807    total: 1m 1s    remaining: 4.95s
925:    learn: 0.0486789    total: 1m 1s    remaining: 4.88s
926:    learn: 0.0486717    total: 1m 1s    remaining: 4.81s
927:    learn: 0.0486567    total: 1m 1s    remaining: 4.75s
928:    learn: 0.0486567    total: 1m 1s    remaining: 4.67s
929:    learn: 0.0486428    total: 1m 1s    remaining: 4.61s
930:    learn: 0.0486347    total: 1m 1s    remaining: 4.54s
931:    learn: 0.0486260    total: 1m 1s    remaining: 4.47s
932:    learn: 0.0486114    total: 1m 1s    remaining: 4.41s
933:    learn: 0.0486028    total: 1m 1s    remaining: 4.34s
934:    learn: 0.0485810    total: 1m 1s    remaining: 4.27s
935:    learn: 0.0485567    total: 1m 1s    remaining: 4.21s
936:    learn: 0.0485481    total: 1m 1s    remaining: 4.14s
937:    learn: 0.0485389    total: 1m 1s    remaining: 4.08s
938:    learn: 0.0485326    total: 1m 1s    remaining: 4.01s
939:    learn: 0.0485190    total: 1m 1s    remaining: 3.94s
940:    learn: 0.0485043    total: 1m 1s    remaining: 3.87s
941:    learn: 0.0484906    total: 1m 1s    remaining: 3.81s
942:    learn: 0.0484815    total: 1m 1s    remaining: 3.74s
943:    learn: 0.0484684    total: 1m 1s    remaining: 3.67s
944:    learn: 0.0484594    total: 1m 1s    remaining: 3.61s
945:    learn: 0.0484474    total: 1m 2s    remaining: 3.54s
946:    learn: 0.0484398    total: 1m 2s    remaining: 3.47s
947:    learn: 0.0484324    total: 1m 2s    remaining: 3.41s
948:    learn: 0.0484188    total: 1m 2s    remaining: 3.34s
949:    learn: 0.0484188    total: 1m 2s    remaining: 3.27s
950:    learn: 0.0484104    total: 1m 2s    remaining: 3.21s
951:    learn: 0.0483957    total: 1m 2s    remaining: 3.14s
952:    learn: 0.0483880    total: 1m 2s    remaining: 3.08s
953:    learn: 0.0483805    total: 1m 2s    remaining: 3.01s
954:    learn: 0.0483613    total: 1m 2s    remaining: 2.95s
955:    learn: 0.0483613    total: 1m 2s    remaining: 2.88s
956:    learn: 0.0483527    total: 1m 2s    remaining: 2.81s
957:    learn: 0.0483453    total: 1m 2s    remaining: 2.75s
958:    learn: 0.0483278    total: 1m 2s    remaining: 2.68s
959:    learn: 0.0483195    total: 1m 2s    remaining: 2.62s
960:    learn: 0.0482972    total: 1m 2s    remaining: 2.55s
961:    learn: 0.0482971    total: 1m 2s    remaining: 2.49s
962:    learn: 0.0482848    total: 1m 3s    remaining: 2.42s
963:    learn: 0.0482810    total: 1m 3s    remaining: 2.36s
964:    learn: 0.0482642    total: 1m 3s    remaining: 2.29s
965:    learn: 0.0482608    total: 1m 3s    remaining: 2.22s
966:    learn: 0.0482587    total: 1m 3s    remaining: 2.16s
967:    learn: 0.0482500    total: 1m 3s    remaining: 2.09s
968:    learn: 0.0482373    total: 1m 3s    remaining: 2.03s
969:    learn: 0.0482161    total: 1m 3s    remaining: 1.96s
970:    learn: 0.0482095    total: 1m 3s    remaining: 1.9s
971:    learn: 0.0481939    total: 1m 3s    remaining: 1.83s
972:    learn: 0.0481842    total: 1m 3s    remaining: 1.76s
973:    learn: 0.0481776    total: 1m 3s    remaining: 1.7s
974:    learn: 0.0481723    total: 1m 3s    remaining: 1.64s
975:    learn: 0.0481654    total: 1m 3s    remaining: 1.57s
976:    learn: 0.0481611    total: 1m 3s    remaining: 1.5s
977:    learn: 0.0481550    total: 1m 4s    remaining: 1.44s
978:    learn: 0.0481407    total: 1m 4s    remaining: 1.37s
979:    learn: 0.0481281    total: 1m 4s    remaining: 1.31s
980:    learn: 0.0481096    total: 1m 4s    remaining: 1.24s
981:    learn: 0.0481066    total: 1m 4s    remaining: 1.18s
982:    learn: 0.0480850    total: 1m 4s    remaining: 1.11s
983:    learn: 0.0480813    total: 1m 4s    remaining: 1.05s
984:    learn: 0.0480682    total: 1m 4s    remaining: 982ms
985:    learn: 0.0480566    total: 1m 4s    remaining: 917ms
986:    learn: 0.0480384    total: 1m 4s    remaining: 852ms
987:    learn: 0.0480320    total: 1m 4s    remaining: 786ms
988:    learn: 0.0480199    total: 1m 4s    remaining: 720ms
989:    learn: 0.0480085    total: 1m 4s    remaining: 655ms
990:    learn: 0.0479959    total: 1m 4s    remaining: 590ms
991:    learn: 0.0479911    total: 1m 5s    remaining: 524ms
992:    learn: 0.0479851    total: 1m 5s    remaining: 459ms
993:    learn: 0.0479672    total: 1m 5s    remaining: 393ms
994:    learn: 0.0479602    total: 1m 5s    remaining: 328ms
995:    learn: 0.0479478    total: 1m 5s    remaining: 262ms
996:    learn: 0.0479352    total: 1m 5s    remaining: 197ms
997:    learn: 0.0479208    total: 1m 5s    remaining: 131ms
998:    learn: 0.0478981    total: 1m 5s    remaining: 65.6ms
999:    learn: 0.0478903    total: 1m 5s    remaining: 0us

정확도:  0.9888059701492538
CPU times: user 1min 58s, sys: 1.57 s, total: 2min
Wall time: 1min 7s

그리드 서치를 통해 나온 하이퍼 파라미터 중 ‘depth’: 13, ‘learning_rate’: 0.1 두 개의 파라미터를 조정하였더니 정확도 약 98%가 나왔다.
속도는 다른 부스팅에 비해 비교적 느리지만 정확도는 가장 높은 것을 확인할 수 있다.

결론

RandomForest에서는 하이퍼파라미터 튜닝 후가 65.6ms 로 더 적게 걸렸다. 이를 분석하기위해 기본 파라미터를 살펴보자.

RandomForestClassifier(bootstrap=True, ccp_alpha=0.0, class_weight=None,
criterion=’gini’, max_depth=5, max_features=’auto’,
max_leaf_nodes=None, max_samples=None,
min_impurity_decrease=0.0, min_impurity_split=None,
min_samples_leaf=1, min_samples_split=2,
min_weight_fraction_leaf=0.0, n_estimators=100,
n_jobs=None, oob_score=False, random_state=None,
verbose=0, warm_start=False)

여기서 우리가 하이퍼 파라미터 튜닝을 위해 넣은 n_estimators= 25는 랜덤포레스트를 구성하는 나무의 갯수를 뜻한다. 기본 파라미터는 100개인데, 튜닝에서 25를 넣어주니 정확도도 더 올라갔고, 속도도 더 줄어듬을 확인 할 수 있었다. 이것은 자료의 양이 많지않아 25개의 트리로 구성하는게 더 빠르고 적합하다 것을 의미한다.

그리고 나는 이 데이터에서 Cat부스트의 성능에 주목했다.
cat부스트에서 주목해야할 점은, 다른 모형들에 비해 기본적으로 뛰어난 정확도이다.
이것은 Cat부스트의 특징때문인데, cat부스트는 수치형데이터보다 범주형 데이터 분석에 더 탁월한 성능을 가지고있다. 그래서 하이퍼 파라미터 튜닝후에는 정확도가 0.988인, 거의 1에 가까운 값이 나왔다고 생각한다

https://ggaggu.github.io/2021/04/15/Tree_Method스터디_범주형/

Author

lhj

Posted on

2021-04-15

Updated on

2021-04-15

Licensed under

You need to set install_url to use ShareThis. Please set it in _config.yml.

Titanic data

전처리

결정트리 생성

GridSearch

랜덤 서치

랜덤 포레스트

그리드서치

랜덤 서치

XGBOOST

gridSearch

랜덤 서치

파라미터튜닝을 하지않은 xgboost

하이퍼 파라미터 적용

lightBGM

GridSearch

랜덤 서치

파라미터튜닝을 하지않은 LightGBM

하이퍼 파라미터 적용

하이퍼 파라미터 적용2

Catboost

그리드서치

랜덤 서치

하이퍼 파라미터 튜닝 전

하이퍼파라미터 튜닝 후

결론

Author

Posted on

Updated on

Licensed under

Like this article? Support the author with

Comments

Links

Recents

Archives

Subscribe for updates