Introducing The ASF’s New Logo Read Now

Board history: S4

ASF Oak Leaf Icon

S4

What the board has decided about this committee or office, and the reports it filed, most recent first. Generated from the published minutes.

These categorised views are best effort. They are read from the published minutes by a program, and what it can attribute to a committee depends on how each resolution happens to be worded — so a page may be incomplete, and nothing here overrides the minutes themselves. The raw minutes are the source of truth.

Please note. The board approves the minutes of a meeting at the meeting after it, and they are published then - so the most recent meeting is not usually here. ASF Members can read the draft in the board's own repository until it is.

[raw] the published minutes, which are the record. [main index]

Reports

2014-06-18 — in the Incubator's report

S4 (Simple Scalable Streaming System) is a general-purpose, distributed,
scalable, partially fault-tolerant, pluggable platform that allows programmers
to easily develop applications for processing continuous, unbounded streams of
data.

S4 has been incubating since 2011-09-26.

No report submitted this month. There have been discussions about retiring the
podling from the Apache Incubator.

2014-03-19 — in the Incubator's report

S4 (Simple Scalable Streaming System) is a general-purpose, distributed,
scalable, partially fault-tolerant, pluggable platform that allows
programmers to easily develop applications for processing continuous,
unbounded streams of data.

S4 has been incubating since 2011-09-26.

Three most important issues to address in the move towards graduation:

1. No development since last release
2. Limited activity on mailing lists
3. Need new committers/contributors

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?

The project is considering retirement

How has the community developed since the last report?

No development

How has the project developed since the last report?

No development

Date of last release:

June 2013: S4 0.6.0 released

When were the last committers or PMC members elected?

March 2013 (Daniel Gómez Ferro)

Signed-off-by:

[x](s4) Patrick Hunt
[ ](s4) Arun Murthy

2013-12-18 — in the Incubator's report

S4 (Simple Scalable Streaming System) is a general-purpose, distributed,
scalable, partially fault-tolerant, pluggable platform that allows
programmers to easily develop applications for processing continuous,
unbounded streams of data.

S4 has been incubating since 2011-09-26.

Three most important issues to address in the move towards graduation:

  1. No development since last release
  2. Limited activity on mailing lists
  3. Need new committers/contributors

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?

  There was a discussion related to retirement but there was some interest
  from the community to continue the project. However there were no
  contributions since last release.

How has the community developed since the last report?

  No activity on dev mailing list over last couple of months

How has the project developed since the last report?

  No major commits since last release. One of the reason

Date of last release:

  June 2013: S4 0.6.0 released.

When were the last committers or PMC members elected?

  Was reported by March 2013 (Daniel Gómez Ferro).

Signed-off-by:

  [X](s4) Patrick Hunt
  [ ](s4) Arun Murthy

Shepherd notes:

  John Ament (johndament):

    The board report reflects my sentiments as well.  S4 seems to be in a
    bit of rut.  I tried kicking off some conversations on the dev mailing
    list, no luck.  It seems like there are at best five active
    participants, between the users list and dev list.  Considering that
    there hasn't been a commit since last board report, it doesn't come off
    as a good sign for me.  I think retirement may be an option to start
    exploring.

2013-09-18 — in the Incubator's report

S4 is a general-purpose, distributed, scalable, fault-tolerant, pluggable
platform that allows programmers to easily develop applications for
processing continuous unbounded streams of data.

S4 has been incubating since 2011-09-26.

Three most important issues to address in the move towards graduation:

1. The frequency of releases has been low in the past. Working to put out
   more releases.
2. The community rise, to be more active and flourish.

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of?

There also was discussion related to “retirement”. Currently people of the
community stepped up and decided continue the project. As result, came up
with building community, providing more patches, implementing new features.
Moreover, Kurtt Lin wrote:

  “Cheer up, guys. I'm seeing opportunities in this stream processing field
  as well as in S4. What I haven’t mentioned is that, S4 had found
  successful applications in our research group even back in the 0.3.0 days
  and now in our start-up. We have built scalable real time search over
  microblogs with some S4 features enhanced (the paper appears in 2013’s
  EDBT proceedings, link is
  http://www.edbt.org/Proceedings/2013-Genova/papers/edbt/a31-lin.pdf); we
  also built a platform on top of S4 for detecting and analyzing real-time
  traffic conditions, processing over hundreds of millions of records every
  day. Now we plan to contribute to the community. I personally have
  created and responded to some JIRAs, but I would like to do more.
  Actually, I am ongoing proposing a design documentation of integrating S4
  with Helix along the roadmap; this would help enhancing the elasticity
  feature for S4. So, I suppose we could put the "retirement" discussion
  aside? Give us the new comers some time perhaps a couple of months to see
  what we can do to make the community healthier. Of course, some of us are
  new; we shall ask for your help and guidance occasionally, please be
  responsive (as what you are doing now), :) You all have done brilliant
  jobs! :) “

One particular problem we have been having is with growth of the community.
Having said that in a recent e-mail thread, we have received very positive
feedback from some developers, so we would like to remain in incubation for
a little longer so that we can get some traction.

When were the last committers or PMC members elected?

Was reported by March 2013 (Daniel Gómez Ferro).

When did the project last make any releases?

June 2013: S4 0.6.0 released.

Two patches are available now, due to 0.7 release 3 of 18 issues was
solved, the rest in progress.

How has the community developed since the last report?

In March, the dev mailing list was reasonably busy (132 messages) and then
that's dwindled to less than 35 per month since March. There was no
activity in git on August at all. A little in July, more in June. There has
been no change in the community. We have been seeing some traffic on the
mailing lists, mostly questions about how to use S4, but the code
contributions have been fairly minimal. However, some developers have
expressed interested in contributing so we expect to start receiving more
contributions in the near future. To grow the community, we plan to have a
closer interaction with other Apache projects like Gora and Helix.

Signed-off-by:

[x](s4) Patrick Hunt
[ ](s4) Arun Murthy

Shepherd notes:

(Dave Fisher/Wave) S4 knows how make releases. Is concerned about community
growth and think that may impede graduation.  I think that they are
actually ready to graduate.

2013-06-19 — in the Incubator's report

S4 (Simple Scalable Streaming System) is a general-purpose,
distributed, scalable, partially fault-tolerant, pluggable platform
that allows programmers to easily develop applications for processing
continuous, unbounded streams of data.

S4 has been incubating since 2011-09-26.

Three most important issues to address in the move towards graduation:

  1. growing the community
  2. verifying (changing?) the name of the project. See PODLINGNAMESEARCH-10
  3.

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware of?

Ant Elder brought up the possibility of graduating. This is being discussed
in the community.

How has the community developed since the last report?

New release created/approved.

How has the project developed since the last report?

Released a new version (0.6.0)

Date of last release: June 3rd 2013

Please check this [X] when you have filled in the report for S4.

Signed-off-by:
Patrick Hunt: [x](s4)
Arun Murthy: [ ](s4)


Shepherd notes:

2013-03-20 — in the Incubator's report

S4 (Simple Scalable Streaming System) is a general-purpose, distributed,
scalable, partially fault-tolerant, pluggable platform that allows
programmers to easily develop applications for processing continuous,
unbounded streams of data.

S4 has been incubating since 2011-09-26.

Three most important issues to address in the move towards graduation:

 1. growing the community
 2. verifying (changing?) the name of the project. See
 PODLINGNAMESEARCH-10

Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?


How has the community developed since the last report?

We have a new committer (dferro).

How has the project developed since the last report?

We kept iterating on the integration with Apache Helix. We also
added major updates to the codebase for improving configurability
(and usability).

We added the sources of the website to our Apache git repository, and
prepared updates to the website and documentation for the forthcoming
release (0.6). This includes migrating the documentation from the
confluence wiki to the S4 website.

We are aiming at cutting a release candidate during the second week of
march for a new release.

Signed-off-by:
Patrick Hunt: [X](s4)
Arun Murthy: [ ](s4)


Shepherd notes:

Things seem to be going well despite the smaller size.  Do you think
you will be proposing graduation before the next reporting period?

2012-12-19 — in the Incubator's report

S4 (Simple Scalable Streaming System) is a general-purpose, distributed,
scalable, partially fault-tolerant, pluggable platform that allows
programmers to easily develop applications for processing continuous,
unbounded streams of data.

S4 entered incubation on September 26th 2011.

Primary issues blocking graduation:

 - growing the community
 - verifying the name of the project. See PODLINGNAMESEARCH-10


Community development since last report:

 We had new users showing interest on the s4-user list, which showed regular
 activity.
 Several new contributors also participated to technical discussions on the
 Jira tickets.


Project development since last report:

 The initial release of the completely refactored version of S4, 0.5.0,
 was focused on providing the basic API and features. We are now preparing
 the next release and focusing on integrations, features and performance.

 Regarding integration, for elasticity, we worked on integrating the Apache
 incubator Helix project, and for resource scheduling, interfaced with
 another Apache project: YARN.

 We also worked on improving performance, which is a key factor for
 adoption.

Signed-off-by: phunt, rvs

2012-09-19 — in the Incubator's report

S4 (Simple Scalable Streaming System) is a general-purpose, distributed,
scalable, partially fault-tolerant, pluggable platform that allows
programmers to easily develop applications for processing continuous,
unbounded streams of data.

S4 entered incubation on September 26th 2011.

Primary issues blocking graduation:

- growing the community
- verifying the name of the project. See PODLINGNAMESEARCH-10

Community development since last report:

We received contributions from new contributors, including core,
communication layer, deployment framework and command set.

Project development since last report:

We had our first release as part of the incubator, on August 16th 2012:
S4 0.5.0-incubating

Apart from being a major refactoring, this release aims at lowering the
adoption bar by clarifying the API and providing a set of commands for the
development and deployment lifecycle.

We also believe this recent release should encourage adoption and
contributions.

Signed-off-by: phunt, bmargulies

2012-06-20 — in the Incubator's report

S4 (Simple Scalable Streaming System) is a general-purpose, distributed,
scalable, partially fault-tolerant, pluggable platform that allows
programmers to easily develop applications for processing continuous,
unbounded streams of data.

S4 entered incubation on September 26th 2011.

Infrastructure issues

No infrastructure issues

Other issues before graduation

1. Still working towards an initial release on Apache with the current
   code (version 0.4 = version 0.3 + checkpointing + bugfixes)

2. Complete the current work on the new version (S4-piper, 0.5), which
   is a major refactoring that simplifies concepts, API, and introduce
   new features such as dynamic deployment and reliable channels.

3. Grow the community. The activity around the project has been low and
   the project simply cannot graduate without substantially increasing
   its activity. We expect to get increased interest once we get a release
   out, in particular of the S4-Piper design. We don't have a release date
   yet, though.

   Additionally, committers have been having internal discussions with
   their companies and outside with their colleagues to attract more
   attention to the project. We have been able to attract some attention,
   but unfortunately that attention has not yet translated into more
   activity around the project.

Project activity:

- 55 Jira issues created to date
- 10 issue reporters
- 5 contributors according to Jira
- Number of users subscribed to the mailing lists: 75 on dev (55 in the
  previous report), 91 on user (72 in the previous report)

Signed off by mentor: phunt
Shepherd: Jukka Zitting

2012-03-21 — in the Incubator's report

S4 (Simple Scalable Streaming System) is a general-purpose, distributed,
scalable, partially fault-tolerant, pluggable platform that allows
programmers to easily develop applications for processing continuous,
unbounded streams of data.

S4 entered incubation on September 26th 2011.

Infrastructure issues

- Migration to the experimental git program completed. No problems reported.

Other issues before graduation

1. Still working towards an initial release on Apache with the current
   code (version 0.4 = version 0.3 + checkpointing + bugfixes)

2. Complete the current work on the new version (S4-piper, 0.5), which
   is a major refactoring that simplifies concepts, API, and introduce
   new features such as dynamic deployment and reliable channels.

   Designed reliable channels; implemented guaranteed message-delivery even
   in the presence of network glitches and partitions using Netty's TCP.
   Pending one bug, this code is up for inclusion.

   Added an Embedded Domain Specific Language (EDSL) to create applications.

   Designed and started implementation of command-line deployment tool to
   make it easy to manage the cluster.

   Design adaptor and inter-application communication.

3. Grow the community

Project activity:

- 45 Jira issues created to date
- 8 issue reporters
- 5 contributors
- Mailing lists: 55 on dev, 72 on user

Signed off by mentor: phunt

2011-12-21 — in the Incubator's report

S4 (Simple Scalable Streaming System) is a general-purpose, distributed,
scalable, partially fault-tolerant, pluggable platform that allows
programmers to easily develop applications for processing continuous,
unbounded streams of data.

S4 entered incubation on September 26th 2011.


* Infrastructure issues
- we are waiting for an answer to our proposal for using an apache git
repository instead of converting our existing code base from git to svn.
This proposal was in response to a call by the infra team for joining
the git testing program.
- we have a volunteer to help with the Python scripts required to be able
participate in the git testing program. His name is Ajaybabu Kalangi.


* Other issues before graduation
1. Migrate code from github to Apache infrastructure and make an
initial release on Apache with the current code (version 0.4 = version
0.3 + checkpointing + bugfixes)

2. learning and applying Apache's procedures (consensus), and
defining/adapting development workflows for the project

3. Complete the current work on the new version (0.5), which is a major
refactoring in order to simplify concepts, API, and introduce new
features such as dynamic deployment and reliable channels

4. Grow the community


* Activity since last report:

1. started using the jira tracker: 30 tickets created and commented so far

2. redirected users and developers from former google groups forum to
apache's mailing lists

3. Migrated web site from www.s4.io to incubator.apache.org/s4

4. Most of the development work has been focused on 0.5

2011-12-21 — in the Incubator's report

DID NOT REPORT.  Still bootstrapping into the Incubator.

2011-11-16 — in the Incubator's report

DID NOT REPORT.  Still bootstrapping into the Incubator.

2011-10-26 — in the Incubator's report

S4 (Simple Scalable Streaming System) is a general-purpose, distributed,
scalable, partially fault-tolerant, pluggable platform that allows
programmers to easily develop applications for processing continuous,
unbounded streams of data.

S4 entered incubation on September 26th 2011.

* Issues before graduation
1. Migrate code from github to Apache SVN and make an initial release on
Apache with the current code (version 0.4 = version 0.3 + checkpointing +
bugfixes)
2. All committers to get ICLA
3. Migrate web site from s4.io to Apache infrastructure
4. Complete the current work on the new version (0.5), which is a major
refactoring in order to simplify concepts, API, and introduce new features
such as dynamic deployment and reliable channels
5. Grow the community


Community
1. Mailing lists have been created. We are already using them.
2. Jira initialized and started to track issues
3. Karma granted for committers with ICLA
4. Confluence Wiki space created
5. IRC channel set-up


Project developments
We have pre-Apache releases on github and our current effort mostly
focused on a major refactoring of the codebase. Goal is to have it ready
soon so that we don't need to support 2 distinct concurrent versions.

Subscribe to ASF Plus One, Our Monthly Newsletter

Subscribe Now