Tuesday, August 29, 2017

Summary for all work

1. Generalized Poisson model (GP-P): GitHub - Status[Merged]
2. Generic Zero-Inflated model , Zero-Inflated Poisson model, Zero-Inflated Generalized Poisson model, Zero-Inflated Generalized Negative Binomial model: GitHub - Status[Review]
3. Generic Truncated model and Poisson Truncated model, Truncated Generalized Negative Binomial model, Truncated Generalized Poisson model: GitHub - Status[Review]
4. Generic Censored model, Censored Poisson model, Censored Generalized Poisson model, Censored Negative Binomial model: GitHub - Status[Review]
5. Hurdle model: GitHub - Status[Need tests]
6. Generalized Negative Binomial (NB-P) model: GitHub - Status[Merged]

3, 4, 5 PR goes as a single PR, but includes very different model. That was done, because Hurdle model based on Truncated and Censored model.

Monday, August 21, 2017

Final week. Summary of all work.

Hi everyone!

Last one week i prepared to merge Generalized Negative Binomial (NB-P) model and tried to fix bug in Hurdle model. For now NB-P model looks pretty good. Current Hurdle model still needs in fixing bug  in Censored model.

Small summary of all work

1. Generalized Poisson model (GP-P): GitHub - Status[Merged]
2. Generic Zero-Inflated model and Zero-Inflated Poisson model: GitHub - Status[Review]
3. Generic Truncated model and Poisson Truncated model: GitHub - Status[Review]
4. Generic Censored model: GitHub - Status[Fix needed]
5. Generic Hurdle model: GitHub - Status[Fix needed(based on Censored)]
6. Generalized Negative Binomial (NB-P) model: GitHub - Status[Pre-merge, last step of review]

So, final week will be spent to make all PR's looks good and to fix all problems.

Monday, August 7, 2017

Second evaluation passed successfully

Hello everyone!

Second evaluation is over. I successfully passed it. Third work period still ahead.

In the previous 2 week was reimpplemented/refactored Negative Binomial model to Negative Binomial  model with p-parameter. Part of the tests was copied from existing Negative Binomial model and missing part was implemented. This model is fully prepared to merge.

Another part of work was Hurdle model.
My first tests shows that previous implementation of Hurdle model didn't work well. One part works well, anoter part works wrong. When i implemented censored model and change wrong part to censored model, hurdle model gave true results. All results i compared with Hurdle model from R package pscl.
Just one problem: how to print these two part as a one model and avoid reimplementing result class. This problem needs to solve in the near future.

Have a good work!

Sunday, July 23, 2017

Result for second evaluation

Hi there! Second part of GSOC was very fast and we are closer and closer to the second deadline. One reason for fast second period is my vacation. I was going to vacation to one week(11-17 july). The vacation was very perfect and i had a good rest. The other reason and the main reason is good work. I have a implementation of Zero Inflated model with Zero Inflated Poisson model. These models are nearly finished. One problem for that model is probit inflation, logit inflation works fine. These models successfully tested. The other part of work was Truncated model and Poisson Truncated model. These models are require more tests, but current tests successfully passed. Based on the truncated models i implemented Hurdle model. As the latest tests showed, this model needs refactoring because results from STATA differ from results of my model. For now i will continue my work. I hope that i pass second evaluation.

Saturday, July 8, 2017

First evaluation, first merge and other success

Hello everyone!

One of the good news: first evaluation successfully passed, as a result i continue work with Statsmodels.
Another good new: first model (Generalized Poisson model) is fully complete and merged.

A little about current model: now i work with Generic Zero inflated with Zero Inflated Poisson and Generic Truncated with Truncated Poisson model.
Zero Inflated Poisson model is at the last stage of readiness (require a few test case). Current Zero Truncated model works with different truncation(in previous blogpost it work only as Zero Truncated model).

And the last good news: i'll go to the one week vacation (11-17 july).

So, let's move on!

Monday, June 26, 2017

Several more models for Statsmodels

Hi everyone!

Two weeks ago i started implementing Generic Zero Inflated model with Zero Inflated Poisson model.

And what's results?
Zero Inflated Poisson model works very well(thanks to Generic Zero Inflated).

Generic Zero Inflated model allows to implement new models very fast. New model require only object of main model(not inflated) and only 1/4 implementation of hessian. Another parts like score, fit, 3/4 hessian is derived from Generic Zero Inflated model and don't required reimplementation.

The next model will be GenericTruncated and TruncatedPoisson. Parts of this model has implemented already. Here is the results:

My implementation:

Truncated model require additional work, because there's some problem with truncation > 1. More little time and it will be fixed.

So, work is go on ;-)

Monday, June 12, 2017

First post at GSoC 2017 with StatsModels and results of first two weeks.


My name's Evgeny.
My project for this summer is adding maximum likelihood models for distributions line Generalized Poisson, Zero-Inflated Poisson, Zero-Inflated Negative Binomial, Poisson Inverse Gaussian and many others(more about maximum likelihood estimation).

Community bonding period was spent to familiarize with this models and look through source code for existing models.

Each model includes the main following components:

  • Loglikelihood function
  • Analytic first and second derivatives(Score and Hessian)
  • Method to predict parameters
  • Method to fit model by parameters
Also each model requires documentation, class to results and a lot of tests(compare result with R or STATA and statistical tests).

First model was Generalized Poisson model(Pull Request).
I have implemented and tested all components already. There is a small problem with the test on CI, but it will be fixed, because model works fine.

Next model was Zero-Inflated Poisson model with Generic Zero-Inflated model(Pull Request). This model hasn't finished yet.

Let's create a great software!