SharePoint 2013 - Distributed Cache: There is a temporary failure. Please retry later.

Following communication issue occurred in my test environment (two-tier farm) after the initial set up and I was forced to reconfigure the Distributed cache.

Events:

1. ErrorCode ERRCA0017, SubStatus ES0006 (RetryLater / CacheServerUnavailable)

Exception in SPDistributedCachePointerWrapper::InitializeDataCacheFactory for usage 'DistributedLogonTokenCache' - Exception 'Microsoft.ApplicationServer.Caching.DataCacheException: ErrorCode<ERRCA0017>:SubStatus<ES0006>:There is a temporary failure. Please retry later. (One or more specified cache servers are unavailable, which could be caused by busy network or servers. For on-premises cache clusters, also verify the following conditions. Ensure that security permission has been granted for this client account, and check that the AppFabric Caching Service is allowed through the firewall on all cache hosts. Also the MaxBufferSize on the server must be greater than or equal to the serialized object size sent from the client.). Additional Information : The client was trying to communicate with the server : net.tcp://SP2013:22233     at Microsoft.ApplicationServer.Caching.DataCache.ThrowException(ResponseBody respBody, RequestBody reqBody)     at Microsoft.ApplicationServer.Caching.DataCacheFactory.GetCacheProperties(RequestBody request, IClientChannel channel)     at Microsoft.ApplicationServer.Caching.DataCacheFactory.GetCache(String cacheName)     at Microsoft.SharePoint.DistributedCaching.SPDistributedCachePointerWrapper.InitializeDataCacheFactory()'

2. No connection could be made because the target machine actively refused it.

Unexpected Exception in SPDistributedCachePointerWrapper::InitializeDataCacheFactory for usage 'DistributedLogonTokenCache' - Exception 'Microsoft.ApplicationServer.Caching.DataCacheException: ErrorCode<ERRCA0017>:SubStatus<ES0006>:There is a temporary failure. Please retry later. (One or more specified cache servers are unavailable, which could be caused by busy network or servers. For on-premises cache clusters, also verify the following conditions. Ensure that security permission has been granted for this client account, and check that the AppFabric Caching Service is allowed through the firewall on all cache hosts. Also the MaxBufferSize on the server must be greater than or equal to the serialized object size sent from the client.) ---> System.ServiceModel.EndpointNotFoundException: Could not connect to net.tcp://sp2013:22233/. The connection attempt lasted for a time span of 00:00:00. TCP error code 10061: No connection could be made because the target machine actively refused it 10.0.0.20:22233.  ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 10.0.0.20:22233     at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)     at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)     at System.ServiceModel.Channels.SocketConnectionInitiator.Connect(Uri uri, TimeSpan timeout)     --- End of inner exception stack trace ---    Server stack trace:      at System.ServiceModel.Channels.SocketConnectionInitiator.Connect(Uri uri, TimeSpan timeout)     at System.ServiceModel.Channels.BufferedConnectionInitiator.Connect(Uri uri, TimeSpan timeout)     at System.ServiceModel.Channels.ConnectionPoolHelper.EstablishConnection(TimeSpan timeout)     at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.OnOpen(TimeSpan timeout)     at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)     at Microsoft.ApplicationServer.Caching.CacheResolverChannel.Open(TimeSpan timeout)     at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Object[]& outArgs)     at System.Runtime.Remoting.Messaging.StackBuilderSink.AsyncProcessMessage(IMessage msg, IMessageSink replySink)    Exception rethrown at [0]:      at System.Runtime.Remoting.Proxies.RealProxy.EndInvokeHelper(Message reqMsg, Boolean bProxyCase)     at System.Runtime.Remoting.Proxies.RemotingProxy.Invoke(Object NotUsed, MessageData& msgData)     at Microsoft.ApplicationServer.Caching.CacheResolverChannel.OpenDelegate.EndInvoke(IAsyncResult result)     at Microsoft.ApplicationServer.Caching.ChannelContainer.Opened(IAsyncResult ar)     --- End of inner exception stack trace ---     at Microsoft.ApplicationServer.Caching.DataCache.ThrowException(ResponseBody respBody, RequestBody reqBody)     at Microsoft.ApplicationServer.Caching.DataCacheFactory.GetCacheProperties(RequestBody request, IClientChannel channel)     at Microsoft.ApplicationServer.Caching.DataCacheFactory.GetCache(String cacheName)     at Microsoft.SharePoint.DistributedCaching.SPDistributedCachePointerWrapper.InitializeDataCacheFactory()'

Cause:

The server was not listening on ports 22233-22266. 

Solution:

1. At the SharePoint Management Shell command prompt, run the following Command stop-spdistributedcacheserviceinstance -gracefull on all cache hosts.

2. At the SharePoint Management Shell command prompt, run the following command remove-spdistributedcacheserviceinstance on all cache hosts.

3. Execute the SharePoint Products Configuration Wizard on all SharePoint servers.

4. At the SharePoint Management Shell command prompt, run the following Command add-spdistributedcacheserviceinstance on all cache hosts.

SharePoint 2013 - Distributed Cache (App Fabric) Error Codes
Manage the Distributed Cache service in SharePoint Server 2013
Original post

Usefull tools: Err.exe (Microsoft Exchange Server Error Code Look-up)

The tool "Microsoft Exchange Server Error Code Look-up" to the contrary to what the name suggests serves to decipher almost any error code.

Here as an example the utilization of the tool deciphering the classic COM error 0x80070005 (Access is denied):

c:\err.exe 0x80070005
# for hex 0x80070005 / decimal -2147024891 :
  COR_E_UNAUTHORIZEDACCESS              corerror.h
# MessageText:
# Access is denied.
  DIERR_OTHERAPPHASPRIO                        dinput.h
  DIERR_READONLY                                          dinput.h
  DIERR_HANDLEEXISTS                                 dinput.h
  DSERR_ACCESSDENIED                             dsound.h
  ecAccessDenied                                               ec.h
  ecPropSecurityViolation                                    ec.h
  MAPI_E_NO_ACCESS                                    mapicode.h
  STIERR_READONLY                                        stierr.h
  STIERR_NOTINITIALIZED                                 stierr.h
  E_ACCESSDENIED                                         winerror.h
# General access denied error
# 11 matches found for "0x80070005"

It returns codes from 172 sources such as SQL, AD, Exchange and so on.

Code sources

Description:

Use the Error Code Lookup tool to determine error values from decimal and hexadecimal error codes in Microsoft Windows® operating systems. The tool can look up one or more values at a time. All values on the command line will be looked up in Exchange’s internal tables and presented to you. If available, informational data associated with the value or values will also be shown.

Microsoft Exchange Server Error Code Look-up
Original Post

Free ebook: Microsoft Project Server 2013 Administrator's Guide

This eBook is a must read for all administratores taking care of SharePoint and Project Server. 

The guide covers all possible product configurations, such as the configuration of resources, permissions ,OLAP cubes and workflows among others .

Microsoft Project Server 2013 Administrator's Guide

Summary:

There are several important tasks that an administrator must manage in Microsoft Project Server 2013 for Project Web App users to access and interact effectively with project data, including

Managing users, groups, and categories.
· Customizing Project Web App to fit the specific needs of your organization.
· Managing workflows.
· Managing enterprise data (custom fields, calendars, views, etc.).
· Managing queue settings for your specific environment.
· Managing time and task tracking.
· Configuring Active Directory synchronization to security groups and resources.

[]

SharePoint 2013 - Workflow suspenso: System.ApplicationException: HTTP 401

O fluxo de trabalho no SharePoint 2013 fica normalmente no estado suspenso com a mensagem de erro "System.ApplicationException: HTTP 401" se o fluxo está configurado para enviar e-mail(s) para um grupo do SharePoint que não permite a exibição da associações.

Mensagem de erro no interface:

RequestorId: <GUID>. Details: An unhandled exception occurred during the execution of the workflow instance. Exception details: System.ApplicationException: HTTP 401 {"Transfer-Encoding":["chunked"],"X-SharePointHealthScore":["0"],"SPRequestGuid":["<GUID>"],"request-id":["<GUID>"],"X-FRAME-OPTIONS":["SAMEORIGIN"],"MicrosoftSharePointTeamServices":["15.0.0.4569"],"X-Content-Type-Options":["nosniff"],"X-MS-InvokeApp":["1; RequireReadOnly"],"Cache-Control":["max-age=0, private"],"Date":["Tue, 15 Oct 2014 14:24:05 GMT"],"Server":["Microsoft-IIS\/8.5"],"WWW-Authenticate":["NTLM"],"X-AspNet-Version":["4.0.30319"],"X-Powered-By":["ASP.NET"]}

Mensagem de error nos logs do SharePoint:

Exception occured in scope Microsoft.SharePoint.Utilities.SPUtility.SendEmail. Exception=System.UnauthorizedAccessException: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))     at Microsoft.SharePoint.SPGlobal.HandleUnauthorizedAccessException(UnauthorizedAccessException ex)     at Microsoft.SharePoint.Library.SPRequest.GetUsersDataAsSafeArray(String bstrUrl, UInt32 dwUsersScope, UInt32 dwUserCollectionFlags, String bstrValue, UInt32 dwValue, UInt32& pdwColCount, UInt32& pdwRowCount, Object& pvarDataSet)     at Microsoft.SharePoint.SPUserCollection.InitUsersCore(Boolean fCustomUsers, String[] strIdentifiers, SPUserCollectionFlags ucf)     at Microsoft.SharePoint.SPBaseCollection.GetEnumerator()     at Microsoft.SharePoint.Utilities.SPUtility.ResolveAddressesForEmail(SPWeb web, IEnumerable`1 addresses, AddressReader func)     at Microsoft.SharePoint.Utilities.SPUtility.SendEmail_Client(EmailProperties properties)     at Microsoft.SharePoint.ServerStub.Utilities.SPUtilityServerStub.InvokeStaticMethod(String methodName, ClientValueCollection xmlargs, ProxyContext proxyContext, Boolean& isVoid)     at Microsoft.SharePoint.Client.ServerStub.InvokeStaticMethodWithMonitoredScope(String methodName, ClientValueCollection args, ProxyContext proxyContext, Boolean& isVoid)

Solução: Permitir a exibição das associações para todos os usuários do SharePoint ou utilizar um outro grupo.

Fontes:

HTTP Unauthorized to /_vti_bin/client.svc/sp.utilities.utility.SendEmail

[]

SharePoint 2013 - Workflow suspenso: System.ApplicationException: HTTP 500

O fluxo de trabalho no SharePoint 2013 fica normalmente no estado suspenso com a mensagem de erro "System.ApplicationException: HTTP 500" se o fluxo está configurado para enviar e-mail(s) para um grupo do SharePoint e o email de saida não está configurado no SharePoint.

Mensagem de erro no interface:

RequestorId: <GUID>. Details: An unhandled exception occurred during the execution of the workflow instance. Exception details: System.ApplicationException: HTTP 500 {"Transfer-Encoding":["chunked"],"X-SharePointHealthScore":["0"],"SPClientServiceRequestDuration":["12"],"SPRequestGuid":["<GUID>"],"request-id":["<GUID>"],"X-FRAME-OPTIONS":["SAMEORIGIN"],"MicrosoftSharePointTeamServices":["15.0.0.4569"],"X-Content-Type-Options":["nosniff"],"X-MS-InvokeApp":["1; RequireReadOnly"],"Cache-Control":["max-age=0, private"],"Date":[" Oct 2014 11:42:51 GMT"],"Server":["Microsoft-IIS\/8.5"],"X-AspNet-Version":["4.0.30319"],"X-Powered-By":["ASP.NET"]} at Microsoft.Activities.Hosting.Runtime.Subroutine.SubroutineChild.Execute(CodeActivityContext context) at System.Activities.CodeActivity.InternalExecute(ActivityInstance instance, ActivityExecutor executor, BookmarkManager bookmarkManager) at System.Activities.Runtime.ActivityExecutor.ExecuteActivityWorkItem.ExecuteBody(ActivityExecutor executor, BookmarkManager bookmarkManager, Location resultLocation)

Mensagem de error nos logs do SharePoint:

Exception occured in scope Microsoft.SharePoint.Utilities.SPUtility.SendEmail. Exception=System.Configuration.ConfigurationErrorsException: Exception of type 'System.Configuration.ConfigurationErrorsException' was thrown.     at Microsoft.SharePoint.Utilities.SPUtility.SendEmail_Client(EmailProperties properties)     at Microsoft.SharePoint.ServerStub.Utilities.SPUtilityServerStub.InvokeStaticMethod(String methodName, ClientValueCollection xmlargs, ProxyContext proxyContext, Boolean& isVoid)     at Microsoft.SharePoint.Client.ServerStub.InvokeStaticMethodWithMonitoredScope(String methodName, ClientValueCollection args, ProxyContext proxyContext, Boolean& isVoid)
Solução: Permitir a exibição das associações para todos os usuários do SharePoint ou utilizar um outro grupo.

Solução: Configurar email de saída para um farm do SharePoint 2013

Fontes:

Configure outgoing email for a SharePoint 2013 farm

[]

SharePoint 2013 - Workflow suspenso: System.ArgumentNullException: Value cannot be null.

O fluxo de trabalho no SharePoint 2013 fica normalmente no estado suspenso com a mensagem de erro "Value cannot be null." se o fluxo está configurado para enviar e-mail(s) para um grupo do SharePoint sem membros.

RequestorId: <GUID> Details: An unhandled exception occurred during the execution of the workflow instance. Exception details: System.ArgumentNullException: Value cannot be null. Parameter name: Input at Microsoft.Activities.Hosting.Runtime.Subroutine.SubroutineChild.Execute(CodeActivityContext context) at System.Activities.CodeActivity.InternalExecute(ActivityInstance instance, ActivityExecutor executor, BookmarkManager bookmarkManager) at System.Activities.Runtime.ActivityExecutor.ExecuteActivityWorkItem.ExecuteBody(ActivityExecutor executor, BookmarkManager bookmarkManager, Location resultLocation) 

Solução: Adicionar usuários para o grupo ou utilizar um outro grupo.

[]

Registrando o Workflow Manager resulta em "An error occurred while sending the request"

O evento abaixo ocorre no momento da registração do Workflow Manager devido a problemas de comunicação entre o servidor do Workflow Manager e do servidor do SharePoint:

Register-SPWorkflowService : Failed to query the OAuth S2S metadata endpoint at URI ‘http://<URL>/_layouts/15/metadata/json/1&#8242;. Error details: ‘An error occurred while sending the request.’. HTTP headers received from the server 

Nos logs do SharePoint:

Microsoft.Workflow.Client.InvalidRequestException: Failed to query the OAuth S2S metadata endpoint at URI ‘http://<URL>/_layouts/15/metadata/json/1&#8242;. Error details: ‘An error occurred while sending the request.’. HTTP headers received from the server – ActivityId: 1b2624e2-42a0-4a93-8c93-2ada0630967c. NodeId: server. Scope: /SharePoint. Client ActivityId : 487d86fd-c961-4d8c-aab2-47269989b292. —> System.Net.WebException: The remote server returned an error: (400) Bad Request 

Verifique o acesso a página http://<URL>/_layouts/15/metadata/json/1 (OAuth Endpoint) do SharePoint a partir do servidor do Workflow Manager. A mensagem no navegador deve ser a mesma - "(400) Bad Request".
A solução seria criar um registro host no servidor do Workflow Manager ou (melhor) um registro do tipo A no DNS do ambiente para o FQDN da página do SharePoint.

[]

SharePoint 2013 e OAuth - Uma visão geral

Project Server - Mensagem "Erro interno ao aplicar a atualização" no histórico das aprovações

O erro "Erro interno ao aplicar a atualização" (There was an internal error applying the update) no histórico das aprovações é comum em ambientes que não possuem  os requisitos mínimos recomendados para os produtos SharePoint 2013 e Project Server 2013 ou em ambientes com problemas de desempenho e ocorre principalmente com projetos grandes.

O seguinte evento ocorre nos logs do SharePoint:

10/03/2014 06:21:35.51 Microsoft.Office.Project.Server (0x07FC) 0x21EC Project Server Task Statusing and Updates av9k Medium Error is: StatusingInternalError. Details: Status approval internal error Attributes:  Microsoft.Office.Project.Server.BusinessLayer.PcsEngine.PcsManagerException: CalcServiceProxy - EndBatchOperation - Exception ---> System.TimeoutException: CalcServiceManager : Timeout when communicating with the worker in sessionId: r2XWq+BmdkWpFUnR5t7Z4yi+7d4vlxHkr2d4K8u/e0ofDJ2i6zAR45P/ABVdTVgg9p05dkr/EeSUDQAVXU1YILfyVh8h9iFKquDFUw7DO6c=System.TimeoutException: This request operation sent to net.tcp://localhost:PORT/GUID did not receive a reply within the configured timeout (00:00:30).  The time allotted to this operation may have been a portion of a longer timeout.  This may be because the service is still processing the operation or because the service was unable to send a reply message.  Please consider increasing the operation timeout (by casting the channel/proxy to IContextChannel and setting the OperationTimeout property) and ensure that the service is able to connect to the client.    Server stack trace:      at System.ServiceModel.Dispatcher.DuplexChannelBinder.Request [...] Name: Project Server Service Application PSError: StatusingInternalError (3116), LogLevelManager Warning-ulsID:0x6176396B has no entities explicitly specified. 7685be9c-ebb6-20c1-33ba-7f54e8451c86

A solução seria corrigir os problemas de desempenho, porém as vezes existe a necessidade de atuar rápido e nós podemos contornar o problema ajustando o timeout "RequestTimeLimits" do serviço "Project Server Calculation Service" por meio do seguinte comando:

Set-SPProjectPCSSettings -Service Application "Project Server Service Application" -RequestTimeLimits 60000

O valor default é 30000ms (30 segundos):

É necessário de reiniciar o serviço "Microsoft Project Server Calculation Service" (ProjectCalcService15) depois da configuração.

Project Server deve processar as atualizações após essa mudança:

UPDATE:

A seguinte consulta identifica as atualizações que não foram aplicadas:

select p.PROJ_NAME, p.PROJ_UID, t.TASK_NAME, r.RES_NAME from pub.MSP_ASSIGNMENT_TRANSACTIONS at
inner join pub.MSP_Projects p on p.PROJ_UID = at.PROJ_UID
inner join pub.MSP_RESOURCES r on r.RES_UID = at.ASSN_TRANS_SUBMITTER_RES_UID
inner join pub.MSP_ASSIGNMENTS_SAVED t on t.TASK_UID = at.TASK_UID
where at.ASSN_TRANS_STATE_ENUM in (4) 
order by p.PROJ_NAME

UPDATE II:

Execute a seguinte consulta para aplicar as atualizações em um projeto determinado pelo GUID identificado a partir da consulta anterior.

Update pub.MSP_ASSIGNMENT_TRANSACTIONSset ASSN_TRANS_STATE_ENUM = 2where PROJ_UID = '<PROJUID-GUID>' and ASSN_TRANS_STATE_ENUM = 4

Abre o projeto no Project Professional e publique o projeto. 

Fontes:

Set-SPProjectPCSSettings

Hardware and software requirements for Project Server 2013