Docs: Document creating new extension APIs

[alamb]

Which issue does this PR close?

Part of #7013

Rationale for this change

@ozankabak's comments on #11404 #11404 (comment) inspired me to write down some thoughts on how the extension process works. This may help @ameyc as we work on proposing new extensions for streaming

There were also some comments at the DataFusion meetup about the challenge of maintaining forks of DataFusion

Also there were some discussions on discord about when to do extension APIs and I was recently quite pleased with how #11207 worked with @samuelcolvin @jayzhan211 and others.

What changes are included in this PR?

Document what I think the current practice creating extension APIs are as well as maintenance of forks.

Note: I am not trying to change the policy, I am just trying to document what the current state is. It would be great if reviewers could confirm / deny this matched their understanding.

Are these changes tested?

CI tests

Are there any user-facing changes?

Docs only

[alamb]

🤔 I wonder if I should try and mention https://github.com/datafusion-contrib here as well

[ozankabak]

I think we should, this comes up fairly often in this context

[alamb]

I pushed a paragraph 294ad54

[ozankabak]

This looks good to me.

I want to note the typical outlook on extension APIs we have: Extension APIs that provide "safe" default behaviors are more likely to be suitable for upstream DataFusion, while APIs that require major changes to built-in operators is less likely. Continuing on the stream processing example, supporting certain stream processing features for built-in DF operators will likely result in a performance loss for run-to-completion use cases. Therefore, we may still end up adding extension APIs to ExecutionPlan for such features, but most likely should leave implementation of such operators downstream.

I'm not sure if we need to add such detail to the doc, but I thought leaving an example here could be helpful to future new collaborators.

[alamb]

I'm not sure if we need to add such detail to the doc, but I thought leaving an example here could be helpful to future new collaborators.

Thanks @ozankabak -- I think it is helpful. I added a paragrap in e571e45 based off your description

[alamb]

I hope to leave this PR open for a few more days to gather any more feedback other community members might have on this content

[ozankabak]

I think we should, this comes up fairly often in this context

[alamb]

I plan to merge this PR tomorrow unless anyone else would like additional time to review

[alamb]

Thanks again @ozankabak and @wjones127

If anyone else has thoughts or suggestions we can make another PR