Sources of data to support the various aspects of projects health
github (contributors, code activities, PRs)
discord (engagement level, responsiveness)
information contributed by project maintainers and members (technology usage)
| Focus | Aspect | Detail | Supporting Data |
|---|
| Community | Growth | new interested individuals and conversion to contributor | - the number of contributors to the code base (github PRs)
- the number of contributors to requirements (github issues)
- the number of contributors to design discussions (discord)
- people working docs (github PRs?)
- page views of the contributor's guide
- meet-up, outside evangelism (people leading meetups in their region)
|
| Diversity | no single organization keeps the project live | - the number of organizations contributing to the code base and roadmap (github PRs, issues, discord discussions)
- contributions to RPC/improvement requests
- should be recorded as percentage of contributions from each org
- language translations of the collaterals
|
| Retention | interesting/useful projects attract contributors, healthy projects retain them | - active contributor longevity (github PRs, discord)
- conversely, the contributors who have become inactive should also be recorded
|
| Maturity | reflects the lifecycle phase a project is in. This gives context to the others, what may be a red flag for a mature project may not be so for a young one | - when was the first commit
- frequency of releases (more mature projects have more regular cadence and have a higher success rate to achieve the cadence)
- docs (high quality docs reflects higher maturity)
- following best practices (security, test coverage, guidance on becoming a contributor etc.)
- are the main committers involving the community in defining the roadmap?
|
| Friendliness | to new contributors/ideas | - number of good-first-issues, help-wanted
- new contributors onboarded
- can new ideas be accommodated, even if that may lead to forking of the code base (how far are core committers willing to go to help)
- is a project defining a consistent way for new ideas and issues to be raised (github issues, etc.)
- Are there PRs without comments.
- time to resolve PRs and issues new vs. core contributors (github)
- no special treatment to core committer's PRs, ideally all PRs should be given equal attention and addressed in order (well defined order taking into consideration of the severity, importance and chronological order) they were created
- are there project meetings in place on a regular basis
|
| Responsiveness | how long until proposed changes (code, design, bug reports, etc.) are given attention? | - time to resolve PRs and issues across the board (github)
- time to respond to questions (discord) - this may not be possible to measure (telling the difference b/w someone making a statement, vs. asking a question that never got responded to)
|
| Code | Usefulness | is the project being adopted by customers and tire kickers? | - usage information provided by customers and developers
- number of questions from clients trying to use the code
- docker pulls
- release binary downloads
- tagged online resources: case studies, presentations, mentorship programs
- amount of research publications it generates
- ask SIGs
- hackathons
- certified vendors?
|
| Production Readiness | is the current code base coherent enough to be usable in a real-world scenario? | - release number (latest is 1.0.0 or later?)
- test coverage
- performance and reliability testing data
- user documentation
|
| Fundamental Metrics |
| - commit rate: number of commits per month etc.
- maybe indicators that would allow us to catch when a project starts to "cool down" or "people are leaving it for other options"
|
| Docs |
| - does it exist
- quality
- LMDWG is creating a badging system for documentation.
|
| Amount of Innovation | how cutting edge? | - ePrint, arxiv
- measures academic interests
|