Note to community. This is a proposal to RESF for funds. In the spirit of open-ness, we want full community transparency and feedback. This proposal has been submitted to the RESF Board but has not been voted on yet. There is time for feedback, comment, and suggestions from the Rocky community. At this time, we (the Rocky leadership) are still trying to figure out the best way of handling proposals so this process will get better. For the moment, I took a lot of inspiration from Fedora as I like how they handle proposals (thank you to everyone at Fedora!). Please feel free to leave relevant thoughts, comments, feedback, suggestions below.
RESF Board Proposal: Increase Rocky’s Sig/AltArch SBC support via purchase of needed tools
Single Board Computers (SBC’s) are a small part of Rocky Linux numbers but the community is growing. There also exists a chicken-v-egg scenario in that many in the SBC community have turned away from Rocky because of a perceived lack of support yet a small demand has left a few dedicated Rocky members doing the majority of the work in supporting the SBC’s. This proposal aims to address this issue by requesting funds for developers, testers, and potentially others in the community as a method to help grow this part of the Rocky community.
Estimated cost: $19,500
This idea has been discussed multiple times in the community. Polls were run for a week mid-December across five social media platforms and results collected. Many have informally volunteered assistance dependent on having hardware to do the work on. This proposal is addressing the immediate needs.
Overview and problem description
Single Board Computers (SBC’s) are complete, or nearly complete, computer systems built on a single circuit board. Generally, they have a CPU, memory, networking, and general I/O features. They are commonly found in embedded systems, portable computers, hobbyist labs, educational systems, and personal computers.
The Raspberry Pi is, by far, the most well known. The Raspberry Pi foundation has done an excellent job in developing a community around their products. Until recent years, it was trivial to purchase a Raspberry Pi and thus there are many in the community who already have a lot of Raspberry Pi hardware but wish for a different OS then the official Raspbian. However, there are still a lot of SBC challengers who bring additional features that the Raspberry Pi does not have, or they provide more or less features at a more or less expensive price. The community for these alternative SBC projects is not as strong.
Author, Software Developer, popular Youtuber, and Rocky supported Jeff Geerling has made multiple comments about how bad Linux support is on alternative SBC’s. See the first minute of this video for one recent example: [WAY faster than a Raspberry Pi—but is it enough? - YouTube]. Jeff is only one voice in the crowd, but he’s got a voice many listen to and he’s only reflecting what the SBC community feels. Far too many of the SBC’s are challenging to get started with for most people who simply want an OS to launch their project from.
There has been an amazing effort by a few dedicated Rocky members to bring Rocky to the Raspberry Pi and a few other SBC’s. However, not all of these members have the tools and resources they need. Further, testing these images is difficult for many on Rocky releases as only a few members have access to the tools and resources needed to sufficiently test. Lastly, when issues do arise in the community there are only a few who can help.
This proposal seeks to address the issues of lack of resources and tools with the hope that this will encourage a more vibrant Rocky SBC community.
Rocky’s SIG/AltArch should “officially” support several SBC’s. As these are not supported by the upstream vendor, this “support” will always be community based inside of SIG/AltArch and other related means of Rocky Linux communication (eg: forums, Reddit, ect). Further, some SBC’s may require newer kernels from Rocky’s SIG/Kernel and thus “break” upstream kernel compatibility adding to “support” existing in SIG/AltArch. To ensure the best chance of support it is recommended that :
- Developers: Rocky will have a few people who have these devices who can ensure packages are compiled and images are built.
- Testers: Rocky will have a few people who have these devices who can run the tests for the Testing Team to satisfy the release checklists (install the image, verify functionality, run a few known-issues tests).
- Community: Rocky will have a few people who have these devices in_such_a_capacity that they can be used to help troubleshoot and debug community issues. This would mean an agreement between the community member and Rocky for a specific duration (EG: one year) that they could use these devices for their own purposes so long as they are willing to be active participants in the Rocky SIG/AltArch community. Ideally, these community members would have higher visibility for community to ask them questions (EG: a pinned topic in forums or headers suggesting posts tag these members in related questions).
The key take away is that we should have people in the Rocky community with each of these boards in each of these three categories. While there can be overlap, ideally the more the better. In the proposals perfect-world-use-case, Rocky would have at minimal five to ten people for each SBC. Two or more willing to assist in the “Developer” role ensuring packages, images, and releases are properly built. Two or more willing to assist in the “Tester” role ensuring tests pass on future releases. And two or more willing to participate in SIG/AltArch community conversations.
In mid-December 2022, community polls were run on Rocky’s Mattermost, Rocky’s forum, Reddit, Mastodon, and Twitter for a week. There were eighty-five responses evaluating fourteen of the most popular SBC’s. This poll did not include the 64bit Raspberry Pi 3 nor 4’s as Rocky SIG/AltArch already produces an image for these SBC’s.
The clear winner was a request for (armhfp/armv7) 32bit Raspberry Pi. Second place was (RISC-V) VisionFive 2. Third place was (ARM) ROCKPro64.
The hardest issue for the first and second place option is that they both require an entirely new architecture to support. While there is community work evaluating these architectures for future Rocky Linux releases, these two will be considered more of a footnote and extra-bonus to this proposal as the work involved in new architectures is quite large.
The RockPro64, the Pine A64-LTS (fourth place), the ODROID-N2+ (fifth place), and the Radxa ROCK 5 Model B (sixth place) are all similar architectures to the Raspberry Pi though they have their own quirks.
An additional SBC has been requested from within the Rocky community, the Nvidia Jetson Nano. As it should run the aarch64 architecture there should be little additional work needed to support and it falls into the same price categories as the others.
The pricing of these SBC’s are between $40 and $150 US dollars per SBC. This proposal uses the $150 price point for calculations as that easily covers the cost of the board, power, and shipping for the vast majority of the boards (other peripherals we can leave to the individual to supply). And the cost difference between the cheaper ones and the expensive ones I think is great enough that the buffer will cover the shipping & power of the expensive ones.
At a minimal
Sherif and Skip currently do nearly all of the development, testing, and support for SIG/AltArch. To purchase a RockPro64, Pine A64-LTS, ODROID-N2+, Radxa ROCK 5 Model B, Nvidia Jetson Nano, and one Raspberry Pi model that they don’t currently have is an estimated $1,800.
At least two Testers are needed to assist on release day: $1,800.
The minimal proposal for these six SBC’s: $3,600
Ideally, Rocky SIG/AltArch would have ten members assisting in one or more of all three roles (Developer, Tester, Community) for each of the six SBCs: Raspberry Pi 3/4, RockPro64, Pine A64-LTS, ODROID-N2+, Radxa ROCK 5 Model B, NVidia Jetson Nano. However, it would not be ideal if the same ten people were responsible for all of these devices. As there have been volunteers in the past who wanted to help but did not have access to SBC’s, this proposal believes this is the best option to bring in more community assistance to both SIG/AltArch and Testing. Thus, there would be a process of identifying Rocky Members and Rocky community to take a smaller number of the SBC’s for different roles and capacities.
Ten members with six devices at $150 dollars each would cost roughly $9,000.
Above and beyond
By far the two most popular votes were for two SBC’s that require different architectures: 32 bit armhfp/armv7 and RISC-V. The prices on each of these is far cheaper then many of the other SBC’s mentioned. Further, many in the community have access to one or the other. The SBC cost is estimated to be under $500 (for two of each) to ensure that more community support is available for both of these boards. However, the cost for developing the architectures is more considerable and out of scope for this proposal at this time. This addendum is added because of the popularity of these boards in the polls reflecting a desire to have Rocky Linux supported. It is not a significant detriment to provide these additional boards to those willing to assist in future porting of these architectures but the immediate ROI will be low on these SBCs.
Shooting for the stars
An additional ten in the Rocky community receiving one of each of the six SBC’s. These would fall under the Community aspect and be “seeds” in the community to both draw in more of the SBC community and assist the Rocky community.
$9,000 for the Preferred core group + $500 for the additional Arch + $9,000 for community seeds + $1000 for overhead for shipping to multiple countries = a rough total of $19,500.
Due to both supply chain issues and the need to find proper volunteers, these purchases do not need to be done all at the same time. SIG/AltArch members could prioritize certain SBC’s. The cost analysis breakdown is not included here as the order in which SBC’s could be prioritized would be a messy matrix. Further breakdowns of price could be done if necessary once a priority list has been made.
Additionally, purchasing SBC’s that currently have no Rocky Linux image for community members decreases the desired goal of sparking excitement for Rocky Linux on these devices. The problem this proposal is trying to solve is the “I have the SBC but I can’t use it” thus getting the Developers access is the highest priority followed by Testing then Community.
At this time, all Rocky Member and Rocky community comments have been positive. Several in the Rocky community have already volunteered to assist in one or more capacities dependent on having a device to work on. While Raspberry Pi 3’s and 4’s are quite popular, there are few that have both in a capacity to assist Rocky though the current lack of supply of Raspberry Pis may prevent Rocky from purchasing the required amounts.
Benefit to Rocky
While the Raspberry Pi community is quite strong, other SBC’s do have such strong communities and many are quite difficult to find information about. Having a stable OS like Rocky Linux would bring many of these SBC owners to Rocky and increase the community engagement as they develop more projects. As Rocky SIG Alt/Arch support grows, it is possible for relationships to develop with the upstream vendors for community referrals and co-operation to resolve issues.
Further, this provides additional opportunity for Rocky mentorship programs. It would identify those in the community willing to assist Rocky in exchange for hardware. These Community members would need to be guided and instructed on how to best participate in the Rocky Linux community in these new roles. This is a perfect opportunity to increase the mentorship required for eventual Rocky Membership.
- Proposal owners: Stack
- Other developers: Sherif, Skip
- Other testing team members: Lukas, Al, Alan
- Release engineering: Neil
- Policies and guidelines: Rocky Members should be given first priority as they already have investment in Rocky. However, broader community engagement would be ideal. We would need to develop a loose guideline for Rocky community outside of Rocky Membership. It should not be complicated legal process. Rather a publicly acknowledged “handshake deal” for participation in exchange for the SBC’s. Guidance on how to properly mentor the community into membership roles would need to be defined.
- Trademark approval: N/A
- Alignment with Objectives: Yes. Rocky Linux objective is stated as “Rocky Linux is an open-source enterprise operating system designed to be 100% bug-for-bug compatible with Red Hat Enterprise Linux®. It is under intensive development by the community.” This proposal falls under the “intensive development by the community” as it seeks to better support the SBC community within Rocky and grow contributions by the community by providing needed resources.
How to Test
As each SBC becomes “officially supported by the Rocky SIG/AltArch” a testing plan will be developed by the Testing Team for each. A general testing plan would cover: installation, primary feature testing, and regression testing against previously known issues.
User experience should be simple and straight forward though this will change slightly for each SBC.
For the current proposal, there are no immediate dependencies. For future considerations of the alternative architectures (32 bit armhfp/armv7 and RISC-V), a much bigger conversation needs to be had with RelEng.
If this proposal fails for any reason, limited support by SIG/AltArch will remain a limited best-effort by a few members with access to the SBC’s.
Will be developed as needed for each SBC.
N/A at this time. Though as support for each SBC grows, identifying these SBC’s publicly would be of great benefit.
As there were many Raspberry Pi devices discussed, the following two links are provided. A complete official list of all Raspberry Pi processors and their corresponding models, and the official store for all currently sold versions of the Raspberry Pi. Not all would need to be collected, but rather a general sampling comprised of both 64bit and 32bit processors.
The VisionFive2 community is still new and as such there are several sources for information:
VisionFive2 information: VisionFive 2 - open source quad-core RISC-V dev board by StarFive Tech — Kickstarter
VisionFive2 Github: GitHub - starfive-tech/VisionFive2
VisionFive2 official vendor list: How to purchase VisionFive 2 - VisionFive 2 （English Forum) - RVspace Forum
RockPro64: ROCKPro64 | PINE64
Pine A64-LTS: PINE A64-LTS V2 Single Board Computer - PINE STORE
ODROID-N2+: ODROID-N2+ — ameriDroid
Radxa ROCK 5 Model B: ROCK 5 Model B — ameriDroid
NVidia Jetson Nano: NVIDIA Jetson Nano Developer Kit